Thompson: Wake Up and Smell the Roses

When economists view schools simply through the prism of an algorithm, it is easy for them to remain oblivious to schools' complexity and diversity. Eric Hanushek in "Waiting for 'Superman': How We Can Save America’s Failing Public Schools," made a big deal about the findings that teachers' value-added vary more within schools than across schools, as if that sound bite said anything about the quality of teaching in those schools.  On the other hand, “Getting Teacher Evaluations Right,” by Linda Darling Hammond, Audrey Amrein-Beardsly, Edward Haertal, and Jess Rothstein showed that Houston’s value-added models are especially unfair for teachers with large concentrations of English Language Learners (ELL) and/or special education students. Even a principal’s decision to reassign a teacher to a different grade could end her career. Hammond et. al described an 8^th grade teacher who had low test score growth, so the principal moved a 6^thgrade teacher with high scores to that class.  Afterwards, the formerly effective teacher’s scores were flat and the previously underperforming teacher had the highest test score growth in the school.  In a letter responding to Darling-Hammond's Education Week Commentary, Anne Evans de Bernard, a principal with 37 years of experience, had a wonderful way of illustrating the "gerrymandering" within buildings, so "any range of scores imaginable" can be achieved.  When teaching immigrants in the 1970s, she endured an annual ritual when scores were released. "The principal would march through the morning lineup and hand a rose to the teachers who had brought in the highest test scores. Invariably, the rose would go to teachers who taught the top-performing students."- JT (@drjohnthompson)image via.

Comments

[Paul Bruno]

I do wonder about the extent to which those reassignment-based VAM gains are legit. I think I'm actually much more effective with CA's 8th grade content than with its 7th grade content. It just so happens that California gives an 8th grade science test but not a 7th grade science test. I've never had a VAM rating, but I wonder if it'd look different for 7th grade and 8th grade.

[john]

Paul,
Think of all the unknowables that are far beyond the ability of any scholar to control for. Every test is a result of political compromises. Each were negotiated separately during different times to fulfill different agendas. So tests were designed to be more rote and/or easier to pass because of the political climate of the time. Others are the opposite. In my district, neighborhood schools have pass rates for some grades that are always in the 80% or above, and the next grade's tests might have a pass rate that is a third of that. Eight grade tests are older and thus more geared to names, dates, facts, and figures. So, are we supposed to give bonuses to all 8th grade teachers and fire all freshmen teachers? The political exigencies are infinite. If VAM advocates had known that before they leaped, we would need to be waiting out the long slow death of theis ill-conceived "reform."

[Paul Bruno]

No doubt, there are lots of problems with using the tests in that sort of high-stakes way. It just seems to me that "the same teacher has different results in different situations" can't be one of those reasons. Wouldn't it be sort of surprising if teachers weren't more or less effective in different situations?