Thompson: Value-Added vs Objective Evaluations

Richard Rothstein's "What Research Really Says on Teacher Evaluation," in the Washington Post, reviewed the large body of social science that explains why value-added estimates are not valid for individual evaluations.  Rothstein then made the common sense point that value-added will poison the well and undermine more appropriate metrics.  He concluded, "one thing of which we can be certain: Armed with knowledge of teacher value-added scores, it will be much harder for principals to observe and evaluate teachers objectively." 

On the other hand, there will be principals with the integrity to resist abuses due to flawed value-added estimates. Award-winning principal, Carol Burris, another guest blogger at Valerie Strauss' The Answer Sheet,  reported that 500 principals returned surveys regarding the first year of New York's value-added ratings of their teachers. Burris explained, "Seventy three percent of respondents said that the 'ineffective' label assigned to some of their teachers was either not a very accurate or an inaccurate reflection of that teacher."  

 Burris cited one principal who wrote: “Two excellent teachers who volunteer to take on my toughest students got an ineffective. Their hearts were broken. So was mine.” Some principals "stated that they would change their teacher’s assignment next year and assign them less needy students so that they could protect these excellent teachers from the ineffective rating."

 And, that brings us back to Rothstein's prediction that some principals will "give high ratings to teachers with high value-added scores in order not to call attention to possible flaws in their observational skills" Others will "tend to offset value-added conclusions in order to save favored teachers who have low value-added."  For instance, the evaluator of a winning football coach with lousy value-added will find a way to put the right numbers on the observation rubric.  Regardless of a coach's instructional effectiveness, many (or most?) systems will do whatever it takes so that the numbers that really count will light up the scoreboard on Friday nights. 

I expect that Rothstein and Burris are both correct and we will see the full range of human reactions to a flawed effort to subordinate human judgment to an algorythm.  The real problem will be the temptation of many (though not necessarily the majority) to Rothstein's question of whether principals will "tend to sink unfavored teachers with high value-added?" 

In every neighborhood high school that I know, there is no doubt about which classes are "dumping grounds," where troubled students are shoved into a side hall in lieu of providing them the services that they desperately need.  Real world, in those types of schools, how many principals will resist the temptation to rid themselves of dissenters by dumping those teachers into classes where it is virtually impossible to meet growth targets?-JT(@drjohnthompson)Image via.

Comments

[Art]

Pretty weak stuff from you and Rothstein.

First, do you know what "objective" means? That is, do you believe that observing teachers is objective but value-added
evaluation is not? It is hardly exaggerating to say that in many places all teachers are rated effective, even when achievement is low and dropout levels are high.

Second, it's true that a group of distinguished scholars issued a broadside against value-added evaluation of teachers. But that was followed by a counterpoint from another group of distinguished scholars that affirmed the use of value-added evaluation. Dueling manifestos make for great theater, but only polemicists rely on only one side in a situation like that.

Third, anecdotes are not evidence. Stories of "excellent" teachers falsely labelled as ineffective because they took on teaching very hard-to-teach children don't speak to whether the teachers really were effective with those children or not. It's mind-blowing that principals would move excellent teachers away from needy students because they're afraid those students would make the teachers look bad. It's like saying a hospital should not put its best doctors in the ICU because patients are really sick there and they don't want to make the doctors look bad. All this does is make the principals look bad, not make value added evaluation look bad. If anything stories and speculations about principal games-playing support calls for applying objective measures such as value-added evaluation.

Fourth, it's false to claim that value added is by its nature prejudiced against teachers whose students start at low levels of achievement. Students who start at low levels can gain as much as students who start at higher levels. Students who start low can wind up at lower levels than students who start at higher levels, but their teachers can add as much value as teachers whose children started at higher levels. There is in fact evidence that students in low-income schools gain as much as students in more advantaged schools and that the distribution of teachers with high value added is similar across these schools.

Finally, it's good that you testify to abuses such as dumping difficult students on athe sidelines instead of teaching them because the public needs to hear that. But what's shocking is that makes you feel sorry for teachers and not the students and that you use this situation as a weapon against evaluating teachers.

[john]

Art,

I wonder where you are coming from. More importantly, I wonder how we teachers can make you the poster child of "reform." Maybe Michelle Rhee will share her ruler with you.

I've read the same research as you. "Reformers" often write the pr blurb claiming that value-add is valid. But, in the text the words are switched, saying that value-added "can" be valid, or "may" be valid or has the "potential" to be valid. Can you name a distinquished scholar who will go on record in a research paper saying that it "is" valid? In fact, can you find evidence that it can be made valid in the highest povery neighborhood high schools? I have read plenty of researchers who expression confidence that future research will bear out their beliefs, but that's a long way from claiming that we have evidence that the numbers already being used to fire high school teachers are valid or can be made valid.

Keeping with that pattern, you write that "teachers (of low-income students)can add as much value as teachers whose children started at higher levels."

Well duh. Can you claim that is true systemically? Can you deny that it is harder to raise scores in classes and schools with intense concentrations of trauma and generational poverty? In fact, have you ever taught one of those classes?

No teacher defends the dumping of the toughest students. Next to the kids and their families, it is toughest on us. Firing good and great teachers because they teach in the toughest schools (made worse by the proliferation of choice), as is true of the general pattern that you seem to advocate of firing teachers based on evidence that may or may not be valid, can't help anyone.