About this blog Subscribe to this blog

THOMPSON: The True NEA Revealed

Candor%20for%20web Who wouldn’t want to be a fly on the wall when Chancellor Joel Klein received a private briefing on the effectiveness of New York City’s school reforms?

Conversely, the leaked, confidential NEA report on six reforms shows that the union's teachers, staff, and researchers are saying the same things in private as they report publicly. The Denver ProComp two-tiered contract caused divisiveness, but the plan’s emphasis on professional development has improved cohesiveness. In private, teachers revealed themselves as less concerned about what was in reforms for themselves, and more preoccupied about their students. A typical comment was, "I would say fewer than 20 percent [of the district’s teachers] understand ProComp, because teachers are more involved in teaching their students than in [determining] their salaries. The problems occurred, for instance, when a teacher has to "jump through hoops to get a raise. It takes my focus away from the kids."

Privately the NEA says what it says in public "Professional development becomes union work, ... Professional development can act as the lever for forging a collaborative relationship between the union and the district."

Confidentially and publicly  the NEA praised the Benwood Initiative in Chattanooga, Tennessee.  Although the project's data was criticized for its lack of transperancy, the resulting bonuses were seen as "icing on the cake."  The candid appraisal was that Benwood's success was attributable to "embedded, focused professional development." "It’s questionable whether the alternative pay system itself contributed significantly" But Benwood "changed the climate of the school; more and more people talk about professional learning."

Unions know that they face generational issues, and the NEA sees that as an opportunity, "the union has had a generational opportunity to do something different." But sometimes, veterans were more supportive of the Denver ProComp because "the experienced teachers who opted-in to the system seemed happier with ProComp than newer teachers, for whom there was no choice."

Policy analysts should study the report for insights into real world complications, such as the problems with applying growth models designed for elementary schools to high schools. Incentive pay plans like TAP are supposed to have a buy-in process and it is a mistake to try to sidestep it. Sustainability of reforms continue to be a concern, and that helps explain the careful reading of research revealed in the leaked study.

A jewel of an appendix includes a "must read" summary of a NIES conference to explain what can and can not be learned by state-of-the-art research. "There seemed to be recognition that teachers have an egalitarian view of their profession ... However, this was not generally viewed as a good thing but as something to be overcome or dealt with when designing pay for performance systems." Lacking educational research, advocates of pay for performance "have theorectically extrapolated from other sectors."

... "Lack of transparency seems to be synonymous with the use of student test score measures, [but] simpler and more transparent measures can often be the most unreliable." ... "Proficiency scores (one of the simplest measures) were cited as the most egregious measure because they are threshold-based and incentivize teachers to concentrate their focus on students with test scores that are just below the threshold."

... "While value-added estimates seem to be superior to measures like gain scores and proficiency measures, they are far from perfect and can also lead to erroneous estimates..... There are several value-added models currently being used by a number of vendors, but the best of these models (the models that provide the most reliable estimates and are based on three or more years of test data) are so technically rigorous and demanding in terms of data requirements that they are rarely used or understood by school districts. ...

"Erroneous performance estimates can lead to misallocated rewards and two teachers that are equally skilled may be rewarded or penalized because of the bias of the estimate being used. The recent trouble in Hillsborough, Florida. (where most bonuses went to teachers in affluent schools), was cited as an example of what can happen when performance estimates are badly biased." - John Thompson, a proud AFT member


TrackBack URL for this entry:

Listed below are links to weblogs that reference THOMPSON: The True NEA Revealed:


Permalink URL for this entry:


Feed You can follow this conversation by subscribing to the comment feed for this post.

Thanks for flagging this paper, John. I don't know why NEA would want to treat the report as "Confidential." It would make more sense to promote the paper.

I second your recommendation of Appendix G, a summary of the 2008 conference at the IES-sponsored National Center on Performance Incentives at Vanderbilt U. Proceeedings can be accessed at


Richard Rothstein's paper, "Holding Accountability to Account" is particularly informative.



Is it just me, or has everyone not realized that the majority of teachers do not generate test data?

For me, this "generates" more than a few problems:

1. If we're all going to be evaluated the same way, and test scores are one criteria, how do teachers who don't generate test scores get the same evaluation?

2. If I'm a 4th grade teacher, but a number of my lowest kids are pulled out for reading and math, who gets the credit, me or the pull-out teacher?

3. If my elementary school departmentalizes and gives me ALL of the 3rd through 5th grade reading classes (which, personally, I would love), then I end up bearing all the burden for the schools reading scores. How do we figure that into the mix? Do I get a 5x bonus?

4. If single year mesures are not that acccurate (and we know they are not) then how do we get multi-year measures when teachers move between grades, subjects, and buildings so often?

5. What do we do about maternity leaves and other times when a long term sub is required?

Etc. Etc. Etc.

All this controversy over tying test scores to teacher evals seems a bit blown up to me when I consider how few teachers will be identified in this regard. It also seems a little odd given the fact that most teachers won't have to worry about this at all since their students don't take high stakes tests.

Once again, in their quest for DATA, policymakers seem to be overlooking the obvious practical matters that go on every day in a normal school. How can pay-for-performance or merit pay schemes be executed with test score data as a signficant component if most teachers don't generate test data?

And why do I seem to be the only person who ever brings this up? Am I missing something obvious here?

Thanks for your wisdom, as always,


Hey, Steve. The "practical matters" you list are just the beginning of a long list. And there is an equally long list of technical matters to accompany the practical matters. You aren't the one who is missing something obvious. While the Federal Government is chasing "better data," available data are being ignored. Go figure.


where's that info on IES longitudinal data? I'm asking because you obviously know more than I do (not because I want to debate someone who knows more than me).
I'll try nnot to steal your insights without attribution.

Also, I don't know anything more than what I've read in the paper and the grant proposal but Tulsa is getting a $15 million grant for preschool, at he same time it seeks $55 million from the Gates, and jumprs into a system-wide 6th through 12 curriculum alignment process through ACT America Achieves. If I know more on preschool maybe i can keep my mouth shut on the Gates sucide-pact, I mean merit pay program.


You ask "who gets credit?" If that's the question, data-driven/informed accountability is one thing. The problem occurs when the question is "Who gets the blame?" and "Who get's the stress?"

I feel for the Tulsa NEA local because they got an offer you can't refuse. (it would increase per capita funding by nearly 30% for five years) The Republicans have said there will be no more pay raises in Oklahoma without merit pay, and last year we were saved by a veto of a bill that would have undermined both teachers unions by turning all schools in charters and through other tricks.. The #1 target was the NEA, and this gets to Steve's question. The viciousness gets visceral. Oklahoma conservatives, at least, start foaming at the mouth at the NEA even though they often have a good relationship with the AFT. (everyone knows it was the NEA that introduced Evolution and drove God from the schools, while the AFT is just a union)

At any rate, the Gates proposal if I recall correctly would link 53% of individual teachers with their students and those who don't meet their expected growth targets will be "exited" from the profession. The same applies to principals. The vendor who designs the VAM will be selected due to their competion in crunching numbers. EVERYTHING in the grant is based on the assumption that mathmematical logic and rational expectations rule.

In a situation like that, the key will not be rewards and punishment. the key will be the peoples' ability to handle the fear and the stress. "Fear is a terrible thing" sings Bruce Springsteen as he describes how it gets into your soul, turning it into Devils and Dust..

Essentially the grant, if approved, seeks to turn every educator into a free agent. I suspect it will work like free agency in sports producing some spectacular teams and prodcing some spectacular meltdowns. I'm predicting two things. The schools that are most troubled, where the students need the most nurturing, will be most likely to give into panic and damage those kids and driving out their teachers. The schools that succeed will have exceptionally honorable principals and it will be their moral character that will be a big factor. Then I anticipate an exodus out of the schools that breakdown just like a bunch of feuding pro ballplayers and talent will seek out school leaders who they trust and respect as human beings.

The longitudinal data I refer to whenever I can is the IES National Center for Educational Statistics (NCES) Early Childhood Longitudinal Study (ECLS) Program. That's a mouthful, and the data are mountainous. The findings have been largely ignored. The portal for getting into the publications is:


The research began with a study of the Kindergarten Class of 1998-99. (ECLS-K)

The Birth Cohort (ECLS-B) began in 2001.

The third study will follow the K cohort of 2010-2011.

The paramount take-away from the research is that it doesn't take 50 states emulating monkeys at typewriters, financed by brain-challenged foundations and federal bureaucrats, to generate reliable evidence about kids, parenting and schooling.


I'll be asking you questions about the databases through e-mail so I don't make public mistakes that are too stupid.

I agree with both your comment to Steve and to me about our need to take advantage of the data that we already have. I don't know that I'd go so far as 50 monkeys at keyboards, but we've always had that problem. My concern is the upsurge of thinktanks and their politicized "research." Given all of the traditional sources of knowledge from government social scientists and established scholarly institutions, I wish that "reformers" had entered into the old-fashioned marketplace of competing ideas rather than going off on their own and ignoring the other sides of the arguments.

The comments to this entry are closed.

Disclaimer: The opinions expressed in This Week In Education are strictly those of the author and do not reflect the opinions or endorsement of Scholastic, Inc.