Millot: Ed Week's Story on Toch's Sweating v. EdSector's Growing

My series on Sweating v. Growing will proceed as planned for next week, but in the meanwhile.....

Debra Viadero's article in today's online edition of Education Week (Study Casts Doubt on Strength of Charter Managers) is worth reading if you are trying to determine the extent to which EdSector manipulated Tom Toch's Sweating draft, and whether it makes any difference. The gap was wide enough for Toch to disown EdSector's authorless Growing report, but Edsector argues that "the sort of editing process it went through would not be something out of the ordinary."

All I can say is that every research analyst should hope it is extraordinary, because if what has happened to EdSectors co-founder and co-director is the ordinary course of business in education policy research, the ordinary staff member is little more than an intellectual serf.

The first bit of new information in Viadero's report is Toch's accusation that EdSector board member, New Schools Venture Fund founder, and CMO Czar, Kim Smith, pressured management to excise financial and other insider information that cast doubt on the future of some CMOs as going concerns.

Kevin Carey's explanation for removing this data is nothing short of bizarre. Education Sector’s policy director, also said the organization chose to trim data about the charter networks supported by NewSchools because it was proprietary information, drawn from internal reviews prepared by the San Francisco group.“This data didn’t meet our standards for quality analysis,” he added, noting that the organization relies primarily on data gathered by the federal government or by its own researchers for its analyses.

My first reaction is to wonder whether this rule would hold if the insider information came from a teachers union. Someone might review EdSector studies to see if this quality standard has been applied uniformly.

My second is that Carey's remarks imply Toch either came by this information by improper means, or  was offered the information with the understanding that it would not be used. I find the first explanation fanciful - no one else has been able to pry much financial data from NSVF since its founding (I've tried) - how could Tom? Moreover, no one put a gun to the heads of the CMO insiders Tom interviewed. The second theory is nonsense. Why would Tom be given access to this information except to write a report? Indeed, after being given this information he could not write a report with findings and conclusions contrary to such information without violating professional ethics.

A tangled web's been woven here; to Shakespeare that suggests deception.

The second bit of news is the absence of Andrew Rotherham. One would expect the organization's co-founder and leader to come to EdSector's defense, or offer an explanation. He is not known to be the shy, retiring type. This leads me to wonder who the chief executive of EdSector is, and whether the organization's internal power arrangement changed when Toch left in June and the board designated Rotherham as Publisher - rather than drop the "co" from Co-director. In the absence of some clear statement from the board we must assume Rotherman is EdSectors legal chief executive, but in fact maybe he's not.

Finally, reporters never have enough space to relay everything said by an interviewee, but like EdSector and the New School Venture Fund, I did have a prepared statement:

Toch's draft questions the plausibility of the CMO "business model"  and the new philanthropy's investment strategy for school reform at scale. The EdSector report is not quite white to Tom's black, but is all the worse for its insidious manipulation of his evidence. The ediface Toch constructed was dismantled in a way that leaves just enough of the problems CMO's face to suggest impartiality, but in the end supports conclusions more in line with CMO proponents preferences.

What's happened here violates any reasonable statement on the professional norms of policy analysis.  Responsible research organizations do not permit managers to appropriate an analyst's final draft, adopt conclusions very much at odds with those in the draft,  edit the draft to support those conclusions, and then hide behind the reputation of the institution rather than accept personal responsibility for the work. That would be sufficient grounds to fire a research director, defrock a professor, and more than enough to deny a PhD.
 
It is almost impossible for this to happen in policy research institutions like RAND, AIR or MDRC. By focusing on the relationship between evidence and conclusions, their formal review procedures for publication go a very long way to protect authors from outside influence.  These organizations charge a premium for their reputations of objectivity, impartiality, and objectivity. Their leaders know that succumbing to the inevitable pressure of  sponsors and clients, or politically ambitious managers, is unethical, devastating to staff morale, and very bad for business. No one study or client is worth the reputational harm.
 
Unfortunately, it takes an egregious abuse of professional responsibility to remind the public, the media and policy makers that all policy research is not equal. Organizations like EdSector support a policy agenda, and they make their living by balancing their funders' policy preferences against a reputation for sound analysis. Sometimes managers make terribly unsound decisions that reveal the conflict of interest to everyone, but they manage the tension everyday, and their sponsors' interests cannot help but color every editorial decision they make.

Comments

[Dick]

The change in the title alone is sufficient "evidence" to support your "prepared statement." A change from "big stuff" to "growing pains" is hardly perfunctory "editing."

A tangled web indeed. A small number of very large "philanthropies" are propping up charter schools financially and "helping" states apply for RttT funds that "assure" the expansion of charter schools. That's leveraging, but it's a business plan with no economic basis.

One would think that the investors would be concerned with the "worst possible case" rather than the rosiest. Something rotten in the state...?

[alexander]

claus von zastrow notes that we haven't heard a peep from andy on his blog about the report or the controversy, and that the MSM has thus far been much slower to jump on the story than it was over the hoxby charter effectiveness study, for example.

good stuff and some interesting comments here:
http://www.publicschoolinsights.org/reporters-interesting-story

/ alexander