Thompson: Did Fordham Accidentally Offer Support for Socio-Economic Integration, Not School Closures?**

It would be easy to read the Foreward to School Closures and Student Achievement, by Fordham’s Aaron Churchill and Mike Petrilli, and prejudge the paper as a similar piece of “astroturf” spin. But, the actual study of Ohio school closures, by Deven Carlson and Stéphane Lavertu, is solid. Their work will be a particularly valuable contribution to education research when it is not misused in support of scorched earth edu-politics where school closures are often used as an anti-union, anti-teacher battering ram.

It is possible to connect the evidentiary dots the way that Fordham does. A careful reading of the study gives as much evidence in opposition to closures as a school improvement strategy as it does in support of Churchill's and Petrilli's soundbites. 

Carlson and Lavertu note, "most of the few existing studies that rigorously examine the impact of school closure have found short-term negative effects (presumably from student mobility) and no discernable long-term effects on student achievement." So, if their study and future research does not conclude that closures produce more benefits than harm, it argues against the Fordham position.

Carlson's and Lavertu's findings "suggest that school-closure policies can yield academic benefits for displaced students so long as there are higher-quality schools nearby that students can attend." Of course, that is a huge "IF." The district schools that were closed were 92% low-income and 73% black. The non-closing schools in the districts studied were 85% low-income and 59% black. Even in this sample, 40% of students were placed in schools that were not higher-performing.

The study also analyzes the closing of charter schools. They were 74% low-income and 73% black.  Non-closing charters were 72% low-income and 54% black. That would be a topic for another post.

Moreover, those statistics are consistent with other research showing the benefits of socio-economic integration in schools and in housing. So, in light of the reappraisal of the Moving to Opportunity program and Robert Putnam's Our Kids, the increases in student performance might not be the result of competition that increases segregation. The gains may be due to a reduction of segregation.

Although you wouldn't know it by reading Churchill's and Petrilli's Foreward, a strength of the study is that it gives two possible baselines to be used in calculating the effects of school closes. The Forward showcases the first, I'd say less meaningful baseline. The disruption and demoralization which accompanies the closing of a school means that the student performance of its last year is not likely to be a fair metric. So, Carlson and Lavertu also generate a second, more "conservative" baseline which is based on student performance during the two years preceding the closure years.

If you only read the Fordham Foreward, you will conclude that three years after district-run schools are closed, its students will gain .073 STD in reading and .041 STD in math. They translate those gains into 49 extra days of learning in reading and 21 extra days of learning in math. But, using the conservative baseline (which seems a lot more reliable to me) reduces third-year estimates .060 or 40 days of learning in reading and to 0.041 or 21 days of learning in math.

If the days of learning estimates are valid (and since the Fordham Foreward gives no reason to doubt them), a casual reader might be more likely to conclude that closures are good policy. I do not have the expertise to determine whether the formula for translating standard deviation into days of learning is accurate or not, so I look forward to other critiques of those metrics.  But, footnote #10 explains, "We warn the reader not to read too much into this metric. ..."

Before School Closures and Student Achievement could be read as evidence for more school closures, these possible benefits must be compared to the costs for students of those policies. Carlson and Lavertu mention this issue and report "there is a significant negative effect when one focuses on the superior, growth-based measure of school quality. ... The quality of the schools that take in displaced students declines by 0.10 and 0.18 standard deviations--for district and charter schools, respectively--before and after absorbing students and staff from closing schools."

It will take additional research to calculate, apples to apples, how the gains of some students compare with the losses of others, but one thing is clear. Even though Carlson and Lavertu study the effects on a large student sample, the numbers of students in the receiving schools - schools that saw a decline - are much greater. So, the data of this study raises the question of whether the harm done to all students is greater than the gains for others.

The authors chose to study a region where student population was dropping, so school closures were inevitable. This raises two issues. First, given the region's economic decline, there is little  reason to believe the Fordham spin that, "Ohio's experience with urban school closures was primarily market driven." Perhaps, as Fordham speculates, "families voted with their feet," but more likely their neighborhoods, like the weaker schools, "withered and eventually died." So, is Fordham arguing for the quicker killing off of low-performing schools or of low-income communities?

Second, we should not the question of the value of research like Carlson and Lavertu when read as an attempt to objectively analyze school closures, as opposed to the spin of competition-driven reformers for their preconceived position. This might sound naïve, but what would happen if reformers read such research for insights into how to best improve all schools, and not as a part of their campaign to defeat unions and traditional public schools?

On the other hand, maybe we should be as brazen as Fordham and other reformers in spinning any and all research as support for our agendas. Then, we could have at least three headlines for Carlson's and Lavertu's work:

1. Fordham's School Closures Work!

2. My reading that School Closures gain produce gains of .060 and .041 standard deviations while producing drops of .10 std* 

3. Or, Maybe we should have tried socio-economic integration instead of market-driven reform.

* Dang if I know what that really means! -JT(@drjohnthompson)

**Editor's note: This post was originally published yesterday, taken down and corrected for spelling and other reasons, and is republished now. Please address any questions to contributor John Thompson @drjohnthompson.

Comments

[Bob]

Hate to be a grammar scold, but the introductory essay in a book or report is a FOREWORD.

[John Thompson]

Bob,
You're as bad as Fordham. when I sent this post out for preview, that is what I was told.

I replied that proper spelling is the sign of a weak mind.

Seriously, I meant to change it, but I got caught up with the precise wording of issues that were brought up, and I forgot.

I'll make the change. Thanks