Thompson: Gates Scholar's Study Casts Questions On Broad Winner*

The  Hechinger Report notes some seriously bad news about the Charlotte Mecklenburg schools, which won the Broad Prize for excellence last week.  The CMS "failed dismally in meeting academic targets for 2011."  And the number of Charlotte schools ranked by Newsweek as among the nation's top dropped from 13 to two. The biggest blow, however, is CMS's rating on the "School Choice, School Quality and Postsecondary Attainment" by Gates scholar Tom Kane and other economists, published by the National Bureau of Economic Research.  The report finds that Charlotte Mecklenburg's massive experiment in school choice has failed to increase performance in students from the lowest-quality schools who won the lottery to attend high-quality high schools.  Instead, the study found evidence in support of the old-fashioned input-driven reforms (such as college preparatory counseling) that the Broad folks and others condemn as the "status quo."  In other words, the central finding of this NBER study calls into question the Broadies' fundamental approach to "reform." - JT (@drjohnthompson)Image via.

*CORRECTION:  The original headline made it seem like the Gates Foundation had funded the study itself, which is not the case.  

Comments

[Parker Baxter]

Please issue a correction. This blog post is completely false. The truth is the exact opposite of what Thompson claims. The report states:

"Our central finding is that students from low-quality neighborhood schools benefit greatly from choice. Lottery winners are more likely to graduate from high school, attend a four-year college and earn a bachelor’s degree." p.3

[john thompson]

You can characterize the central finding any way you want. This is the latest in a series of reports where the evidence and the spin are contradictory. A fair characterization of the central finding is that the program failed to improve student performance. The big implication of that finding, as the report explains, is that it is unfair to hold schools accountable for factors over which they have no control. So, the report contradicts the fundamental assumption of the Broad School, of the Gates MET, of NCLB, and data-driven accountability. I think that that should have been characterized as the finding that is most important for national policy.

After all, there was no claim that choice improved capacity. Yes, if magically we had more high-quality schools, students would benefit greatly. The more important issue is how do we build capacity as opposed to rearrange the deck chairs. The truth is that the implication of the report is that old-fashioned input-driven capacity-building is what works. The report explained:

"Because the choice schools were often magnet schools, with specialized programs such as career academies, arts education, and intensive college prep, the benefits could come primarily
from improved student engagement in high school. It is possible that having demographically similar but more able peers led to increased student learning and engagement inside the classroom. Better peers could also have an impact on behavior inside and outside of the classroom. The impacts on mediating variables as early as 9th grade are inconsistent with a scenario where “better” schools are simply more organized at getting students to surmount administrative hurdles such as accumulating enough credits in the right high school courses, or filling out college applications and financial aid forms."

In other words, the report provides support for the conventional wisdom of teachers - the wisdom that the accountability movement condemns as "the status quo."

[Parker Baxter]

No, you cannot "characterize the central finding any way you want" unless you don't care about being accurate and only care about pushing your agenda. I am not debating with you about whether or not the study supports any one approach to school reform. I am simply noting that you are completely mischaracterizing the actual results of the study. Instead of trying to change the subject, you should correct the post. Your falsehoods are embarrassing.

Here is your description of the study's "central finding":

"The report finds that Charlotte Mecklenburg's massive experiment in school choice has failed to increase performance in students from the lowest-quality schools who won the lottery to attend high-quality high schools."

And here is the author's statement:

"Our central finding is that students from low-quality neighborhood schools benefit greatly from choice. Lottery winners are more likely to graduate from high school, attend a four-year college and earn a bachelor’s degree. They are about twice as likely to earn a degree from an elite institution such as UNC-Chapel Hill or Duke. For these students, the impact of winning the lottery is large; closing nearly 75 percent of the black-white gap in high school graduation and 25 percent of the gap in bachelor’s degree completion. We also estimate the impact of winning a school choice lottery on high school test scores, graduation, and other school outcomes. We find no impact on 9th grade test scores, but relatively large improvements in other school outcomes such as grade point average, math course-taking, and absences."

Can you seriously claim that your original post is an accurate description of the study's findings?

[john thompson]

Let's look at the facts, and let readers decide what the central findings are. A high-profile district is being showcased as an exemplar of data-driven, market-oriented reform. In fact, that district is pioneering a dramatic expansion of standardized testing. A co-author of a study is the director of the Gates MET project. And if the study found that Reading scores increased, don't you think that would be the central finding?

A major study of that district documents real benefits for a hundred or so students.

While we can all celebrate the benefits for those few, don't you think that most readers, across the nation, are interested in lessons how we can use the new evidence to help more kids? And what were those lessons?

Firstly, it raises the question about school and/or teaching quality. Were those schools that were identified as low-quality determined to be so because their teachers were not doing their jobs well and thuse not adding value? No evidence was given for that.

Were they low-quality because they did not provide the same supports? Evidence was given that those supports might have explained the benfits. That raises the question of whether resources could have been better spent giving those schools the supports that the magnet schools enjoyed.

Were they low-performing due to peer pressure? Evidence was given for that. And of course, that is the point that traditional reformers have been trying to make.

And that gets to what I argue is the most important point of the paper. It helps explain why it makes no sense to hold schools, or teachers, accountable for circumastances beyond their control.

The authors of that report, to their credit, volunteered that statement.

The authors chose to tell their narrative the way they chose to tell their narrative. You may not be arguing whether the study supports any one approach to school reform, but the authors knew that that huge conflict exists. I am being equally accurate, and fair, in arguing how I see the evidence in terms of that debate.

I apologize, but I do not know you and so I don't know where you are coming from. Are you coming from Charlotte Mecklenburg? If so, I'd think you would want to rethink that district's approach.

If you are looking at the issue from the perspective of choice as a national policy, I'd think you'd be interested in the evidence that socio-economic integration might be the goal of choice approaches.

If you are looking at it from the perpective of Bill Gates and Eli Broad, I'd think you would see this as evidence that you've pushed too fast down the wrong path, and you might want to listen more to actual practioners.

[web design Landon]

It raises the question about school and/or teaching quality. Were those schools that were identified as low-quality determined to be so because their teachers were not doing their jobs well and thuse not adding value? No evidence was given for that.

Were they low-quality because they did not provide the same supports? Evidence was given that those supports might have explained the benfits. That raises the question of whether resources could have been better spent giving those schools the supports that the magnet schools enjoyed.

[Alexander Russo]

correction: the original headline stated that the study was funded by gates, which is note the case. my apologies. / ar