Thompson: Correcting TNTP's "Widget Effect" Stats
The New Teacher Project has finally corrected apologized reconciled its ubiquitous report's criticism of the American Federation of Teachers' Toledo Plan, admitting that crucial figures were off by 200%, if you accept the TNTP's definition, and off by 530% if you accept the AFT's terminology.
Even when drafting "The Widget Effect," the TNTP should have known that one of the strengths of Toledo's peer review and mentoring program is that it removes ineffective teachers through resignations and other means, thus avoiding litigation. So it came as a shock when the TNTP claimed that the Toledo Plan only removed .7% of probationary teachers over a five year period. In fact, 12.9% of teachers in the plan were removed from the classroom in 2009, and over the last five years 8.5% of interns were non-renewed, terminated, or resigned.
Read below for more about how the Toledo Plan compares to other districts' removal rates and the dispute about the numbers used in The Widget Effect.
This compares with the percentages of 2008 probationary teachers removed from the classroom in Sycruse (9.7%), Rochester (7%) and Montgomery County (10.5%), not to mention the TNTP's own figures of 37% of peer reviewed teachers in Minneapolis being exited. And this illustrates another virtue of the Toledo Plan; it can be expanded to address teacher quality of veteran educators, and it can even be expanded to a method, as in The Grand Bargain, to utilize test score growth.
The Toledo Plan mentors and helps retain new teachers; it can dismiss a greater or lesser percentage of teachers, depending on need and capacity; and it can avoid the huge litigation costs that will be inevitable if districts use Value Added Models to indict teachers as ineffective.
I cannot judge whether the TNTP's or the AFT's definitions of who should be counted in Toledo are better, and to their credit the TNTP now intends to publish both. Even by their own figures, however, three times as many teachers were removed from classrooms in Toledo than they reported in "The Widget Effect." Perhaps it is unfair to ask what did the TNTP know and when did they know it, in regard to their public announcements. Apparently the Toledo Federation of Teachers was slow in getting the accurate data to the TNTP. On the other hand, in May of 2009 - one month before publishing "The Widget Effect" - the TNTP issued another report in Minneapolis which shows that they understood the nature of peer review. Then in October they participated in a conference call which reconciled the erroneous data.
A month later, however, the TNTP's Tim Daly again attacked the effectiveness of peer review in Steve Sawchuk's excellent article that explained why so many teachers are so proud of peer review.
The Minneapolis report illustrates the TNTP's methodology. Nowhere in the Executive Summary is there a hint that the full document provides evidence in favor of the effectiveness of peer review, or any indication that their recommendations include an awkwardly phrased endorsement of the program.
As usual, the report includes polling results that are completely unsurprising, and selectively uses those numbers to illustrate, though not substantiate, their arguments. The TNTP reports that only 29% of principals are satisfied their district's teacher evaluation system, which is hardly a revelation since 27% of those principals are satisfied with their evaluations. Only 43% of principals think they should spend 1/2 of their time observing classrooms, and only 11% invest that much time. Only 15% of teachers think that principals are best equipped to evaluate them while 65% of teachers believe that their colleagues should conduct evaluations. So, the TNTP provides more evidence that a big reason for our incompetent evaluation system is that principals do not have the time or training to conduct evaluations, and a sizable majority of teachers think that peer review is the best system. You would never get a hint of that evidence, however, by reading the TNTP's public relations statements.
The key issue is whether the TNTP will mend their ways. They have gotten great milage from misstating the facts of the New York City Absent Teacher Reserve, and spreading those inaccuracies through The New Yorker and middlebrow periodicals. But as states seek to act on their RttT promises, facts will be more than stupid things. They will be evidence in procedings with the force of law. Perhaps this reconciliation will allow both data-driven and traditional reformers to move ahead as partners, not as plaintiffs. I would have loved an apology, but I would be satisfied if the TNTP would heed the concept of "the loyal opposition," and see teachers and our unions as opponents not enemies.