THOMPSON: Getting the Facts Straight About the Realities of Urban Education

I’ve always respected the work of Robin Chait and Raegen Miller, so I’m dismayed by their report "Getting the Facts Straight on the Teacher Incentive Fund." They admit that the "TIF requires that compensation systems consider gains in student achievement ... and that their grants "are explicitly structured to help schools transform their compensation ..."

Is it any reassurance that TIF’s efforts to stimulate NATIONWIDE, SYSTEMIC change do not allow test scores as the "sole" method of evaluation? 

Chait and Miller cite an example of a district that uses data properly, and some incentive systems like the Teacher Advancement Program that make sense.  But teachers must protect themselves against the invalid use of data.  If Florida extended its merit pay formula (of 60%) to evaluations, a teacher in a tough school would start every year with "two strikes against him."  If a district used TAP's bonus model for job retention decisions, a teacher with challenging classes would be like a basketball player going into every contest with "three fouls on her."  Teachers could never enjoy any peace of mind.

Primitive and unreliable statistical models based on primitive and unreliable standardized test scores, if applied substantially to individual evaluations, represent a loaded gun aimed at the heart of the teaching profession. A flawed bonus system would just be a mistake, and mistakes happen.  Adopt an incompetent evaluation or tenure model and no self-respecting professional would remain in a high-challenge school where it often takes a truly heroic effort to keep scores from falling even further.  Would teachers of English Language Learners, Seriously Emotionally Disturbed and mentally ill students, or the most traumatized and behaviorally challenged students be willing to trust their careers to the wisdom of social engineering?

Would "reformers" with no practical experience in the urban schools seek to use the same growth model for Algebra I as for Algebra II, not knowing that the challenge of those two classes could not be more different? It is one thing to raise test scores with inner city juniors and seniors who survived until they reached higher math, while ½ of their classmates in Algebra I dropped out, as opposed to meeting statistical goals with rowdy freshmen who frequently have an abysmal enrollment and attendance rate. This is especially true when schools refuse to enforce their attendance and disciplinary policies. 　

How’s this for a counter-proposal? Ban "credit recovery programs," outlaw the "working off of absences, and provide strict sanctions for pressuring teachers to just "pass on" students who have not met standards, and then we could talk. Put an end to curriculum narrowing, excessive test prep, and gimmicks that may boost decoding skills, while driving down reading comprehension, and then we can talk about the role of student performance data in teacher evaluation.

Or seriously, outlaw evaluation models that may be competent for calculating incentives from use in individual performance evaluations or tenure, thus threatening the very careers of educators

Doctors are allowed to use medicines and technologies for purposes for which they have not been approved.  Even in our inefficient health care system, however, we have systems to control incompetent experimentation.  We can not fault innovators for loving their new tools.  But we need institutionalized systems to deter educational malpractice.  For a concise guide of how that could be done, check out Sherman Dorn's "Bayesian bump." - John Thompson

Comments

[Charles Barone]

John Thompson vs. CAP and the civil rights community, including icons like Bill Taylor and Roger Wilkins. All in one week.

[john]

What can I say?

I punch above my weight.

Stick with me, Charlie, and I can help you hone your killer instinct.

Seriously, I was counter-punching, responding to attacks on my union. What I'd really like is not a response to my original jabs, but to the peace offerings at the end of each post. This often comes down to the difference between data-driven accountability and data-informed accountability. Teachers and unions have to fight to the end against data-driven accountability, but how about Sherman Dorn's "Bayesian bump" that would make a great compromise? Sherman will correct me if I over-simplify, but I think he means that tests scores could be used to supplement, complement, document, refute, and/or explicate the effectiveness of a teacher. It would be human judgments, not mindless statistical models that drive the process.

And I'd still lke your thoughts on the un-footnoted paragraph by Taylor on agreeing to not forcing teachers to transfer to reach equity. If he's accurate, it sounds like a perfect compromise. We agree that districts can not force or deny teacher transfers when protected by collective bargaining agreements and we form a united front for comparability. We teacherswould be idiots to turn down additional money for poor schools, which would also mean more money for union members if we didn't lose protections against being moved around like chess pieces.

[charlie]

I think you missed the point of the Chait and Miller piece. They're talking about a mix of things, just like they do in Denver.

Who's proposing forced transfers? That's a red herring. Don't take the bait. : )

[john]

No, Chait and Miller were responding to criticisms of the TIF. I saw some of their objections as strawmen, but I ignored them. I was focusing on the point that teachers and unions have to fight.. So again, in this piece the CAP was pushing for a set of tools and policies that could transform compensation systems and evaluations across the nation. If they just want to change incentive systems we can agree or disagree but that is not a mortal threat to the profession.

But here's the problem. If you sweep away some current laws and contract provisions, then the same tools and policies that are used for awarding bonuses could then be used to fire teachers. Work with us for legal protections so that statistical models that may be accurate enough for awarding bonuuses, but that are not nearly accurate enough to end a teacher's career, are not used for individual evaluations or tenure, then we could collaboratively implement policies that use data appropriately.

The Comparability issue was raised in the other post that you cited. How can my union's leaders compromise when you reformers are so flippant about honoring contracts? M. Roza repeatedly tap dances around this issue, and then she wants to ignore contractual obligations in an even worse manner and fire teachers in an economic downturn in violation of contracts. If forced transfers are a red herring, why not sit down with the unions and craft regulations so that equity has to be achieved without them?

I really wish I could get an answer to these questions. If we had these sort of assurances and the unions didn't respond, then there are plenty of us that would put pressure on our leadership. I keep hoping that most "reformers," if they understood the hard facts of why teachers have to stand firm on some issues, would work with us on data-informed accountability and innovations like Denver..

[Steve]

John,

As you know, this is not my area of specialty, but after reading all of these fine thoughts, a few questions came to mind:

1. Would we be arguing so much over teacher bonus formulas if teachers were paid like other professionals?

2. Would we be arguing so much if we simply used human judgment (like most organizations do; for example, you boss sets your bonus) when it came time to evaluate and compensate?

3. Aren’t all teacher bonus formulas fundamentally flawed because the teacher can’t see his or her progress over time as it occurs but rather must wait until the data from test scores and other evaluative instruments have been gathered? (By contrast, a car salesman working on commission knows where he stands relative to a bonus on almost any given day. This helps him make good decisions that are ideally aligned with the company he works for. How do teacher bonus plans do that? And if they can’t, what use are they really?)

All of this is just my way, once again, of trying to promote simple, sensible solutions that are widely used and already in place all over our society. The toughest arguments that seem to crop up between teachers and the rest of the world all seem to stem from aspects of being a teacher that are different from being a part of the rest of the working professional world. Do teachers think they actually ARE that different? Do they WANT to be that different? Or are these differences just a series of historical holdovers?

Finally, firing someone is a serious decision. And if you’ll notice, even in fields like medicine, law, and professional sports, where firing is always on the table every minute of the day, firing is not taken lightly and professionals don’t shiver in their skivvies every day from 9 to 5. Why? Because firing is very costly to the organization. For example, principals have no job security or union protection, and many are barely competent. Yet few are fired because the cost of replacing them is too high, and the likelihood of finding someone better is too low.

The more we strive in education to make compensation, hiring, and retention systems match those that already exist elsewhere in the professional world, the sooner we will find ourselves in fewer arguments. And as soon as we stop arguing, we can start helping kids.

I am not anti-teacher. My mom was one and I was almost one. My mom and our family benefited dramatically in the 1970s from collective bargaining. But in the 80s and 90s, my mom didn’t seem to get much for being part of her union, and she grew progressively more uncomfortable with being a member of a group that seemed to stand for so many things she was against. She craved what she called “the real world of work” and took great pleasure in a job as a testing coordinator for a local law school shortly after she retired from the classroom. She got a nice hourly wage, loved the people she worked with, and had no job protection or bonus structure whatsoever. Neither did anyone else at the school – except the professors, of course. Best work experience she’d ever had. She even asked for a raise at one point and got it! She was so proud of herself.

So I’m not anti-teacher. What I’m anti is arguing about things that don’t make much sense even when resolved. Deciding on a set of criteria for a teacher bonus plan won’t change much. Neither will holding onto structures like our current approach to teacher evaluation or the Byzantine processes a district must go through to fire someone. And although I know most teachers disagree with me on the need to retain tenure or continuing contracts as a staple of one’s employment package, I haven’t met at teacher yet who actually said that during the time early in their career when they didn’t have tenure or a continuing contract that they were all that freaked out about not getting one. Obviously, hundreds of thousands of public school teachers haven’t taught long enough to be granted tenure or continuing contracts. These are, in general, our least able and least experienced teachers. So why don’t they get fired in droves? Because, again, it’s costly to the organization.

So what’s the point of all this?

ONE SIMPLE RULE: Whatever structure you choose for firing, hiring, and compensation has to be one that is known in other parts of our society to promote healthy organizational relationships and strong individual performance.

Don’t experiment on teachers. That’s just as wrong as using things like NCLB and standards to experiment on kids.

And teachers: Don’t continue to experiment on yourselves with a compensation system that may have served you reasonably well 30-40 years ago but certainly doesn’t have much of a chance of withstanding popular trends and what will surely be more calls for ways to recruit and retain higher quality teachers in the future. Get out ahead of the game for once. Be aggressive in designing a package the maximizes your value based on your REAL VALUE AS FACILLITATORS OF LEARNING, not on body count or some set of historical benchmarks which may not have even made sense when they were initiated. If you don’t like the way things are now, (and I don’t think most teachers do) put something else on the table. Be as radical as you like. Just follow the rule that whatever you come up with has been shown to work some place else.

If we all stick to the ONE SIMPLE RULE, things will work out well for everyone. And especially for kids.

[john]

Steve,

I don't care one way or another on performance pay, so for me its a good point to concede in the spirit of collaboration. If younger educators want performance pay, that's fine with me and if it advances collaboration that's all to the good.

I also want the union to continue to move forcefully to negotiate new systems where we can efficiently remove bad teachers. And I won't oppose Duncan's turnaround strategey of replacing entire schools. We need all types of turnaround strategies and we must take some risks and accept some pain.

But think of how much more difficult it is to compromise if we don't believe in the rule of law and respecting contracts. Sure, under circumstances contractual rights get trumped, but we still need to respect those agreements except in extreme cases.

Every principal has an annoying teacher. If data-driven accountability took hold, most principals would not disrespect the spirit of the law to fire those annoying teachers. But how many would? If you have an effective teacher who questions policy, most administrators would not transfer that teacher to a class where nobody but nobody would meet statistical targets. But how many would?

If the federal government prohibitted store owners from locking up at night, most citizens would not become criminals. But how many more would?

George Soros explanation of part of the financial collapse is central. If you have ten bottles of water and only one is poisoned, all are worthless. Who would invest their souls in a career where it could be destroyed at any time by a statistical quirk, when you see your friends' careers destroyed unfairly, and when nobody knows whether they could be the next arbitrary victim?

[Steve]

John,

Wow, I had no idea teachers felt this way. I also found your explanation of the consequences identical to my own personal experience of working in the world outside of school. I've been almost fired many times for speaking up. And the one time I did get fired, I sorta deserved it for screwing up a major project (though there were extenuating circumstances, of course.) And, yes, it was painful. But it was probably also the most valuable experience of my working life.

What I know is that you're right: a few bad principals will make a few bad decisions. But isn't that just like a few bad business owners or managers making a few bad decisions? Most of the world works under these same conditions and the world of work seems to function pretty well most of the time. I'm not sure Soros' water analogy is valid because he's talking about a "tainted" product. Whereas there is no reason to expect that principal B is tainted just beause principal A is incompetent.

So while I am in total agreement with you about what would probably happen, I'm wondering -- as I so often do! -- what would make this situation any worse than the current job situation for most folks in the world outside of teaching?

My hunch is that it would have a benefit in most cases. There's a healthy tension in healthy tension between employee and manager. Both have quite a lot at stake in having a good relationship. I've been on both sides many times and I can tell you that I find scant advantage in being the boss -- especially when my employees are actually talented people. If my employees are not talented people, I have more options. But still a lot of work to do. Is this what teachers are secretly worried about? That most are not talented people? This would certainly help me understand why so many feel the way they do. But, of course, this is not something people are likely to talk about.
'
Now here's where I think we might hit some hard reality in teaching. What if -- and I'm saying purely "what if"-- it were found that 10% of the worst teachers in the US were no better than first year teachers? Then districts would be silly not to identify the worst performers and replace them with newbies. But here, too, there's substantial risk and cost to management as new teachers bail out of the profession at astonishing rates (33% in two years; 52% in 5 years). So in order to pull this off, management has to KNOW it can find enough new teachers -- and keep replacing them year after year. In theory, this leaves management no better off than it was with 10% of failing teachers. So, again, even in this scenario, they are unlikely to embark on such a huge effort for such a small potential gain.

Finally, of course, bad principals who made back hiring and firing decisions would quickly wind up with failing schools. And I certainly suggest that they be replace just as quickly and easily as poor teachers. What's more, recruitment and retention would soon become valuable skills that principals might actually develop as they become part of the known universe of the profession. That would be a boon for everyone. (Currently, the 150 or so principals I have worked with are uniformly awful at recruitment and retention. Only one or two have been savvy enough to build a good staff over time.)

Finally, in a fluid employment situation, would anyone's career be ruined by being let go? Or by moving to a new building? Or a new district? Such moves go on all the time in the world outside of school where most of us are now slated to have 10-15 different jobs over the time of our careers. Much as I have hated looking for work, I have to admit that it has always made me a better person. And when I see myself as an entrepreneur, I am strengthened even further by taking a higher degree of responsibilty for myself and for my work.

Again, I'm not saying this should apply only to teachers, or that special formulae should used against teachers. I think teachers, and everyone else in schools, should play by the same rules that most everyone else does in the world of work. And I say that not out of any sense of fairness but merely out of the sense that "normal" employment actually works pretty well to improve organizations and people over time.

Thanks, as always, foryour perspective,

Steve