About this blog Subscribe to this blog

Evaluation: A Revolt Against The "Randomistas"?

Flickr Meghan Carnavalet In These TimesAre you an unapologetic "randomista" -- an advocate of randomized controlled trials as a way to mesure the impact of social interventions -- or do you dare to consider some of the drawbacks behind what's commonly called the "gold standard" for evaluations in edreform circles? This recent Slate article by Joshua Keating might help you decide: Randomized controlled trials: Do they work for economic development?.

RCTs are increasingly popular with the public and policymakers -- with TED Talks and New Yorker profiles -- but also expensive and difficult to implement, strip away key contextual information, and lack generalizability. They're also over-adored by politicians and journalists. "Media and policymakers tend to overstate the conclusions of randomized controlled trials," according to Keating.

The piece focuses on evaluation of international development but also contains an interesting story about randomized trials in education improvement efforts in education. Specifically, it tells the story of an attempt to figure out whether more textbooks or other interventions worked best in improving education outcomes. It turned out they didn't.  Better teaching strategies and health care did. Other examples cited in the piece include one that found school uniforms helped prevent teen pregnancy more than sex ed. Very Malcolm Gladwell.

I don't personally believe that research can prove things in social sciences, in part because of evaluation limitations (and time delays, etc.) but also because of the tendence of people to disbelieve research findings that don't comport with their beliefs.  If something's proven but the proof isn't accepted widely, then -- for a time at least -- the issue remains unsettled in the public debate.  That's why my research category on this site is titled (Who Cares What) Research says.  I feel a bit anti-intellectual in writing that, but I only mean to be pragmatic.

Image via Flickr.



Feed You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.

Disclaimer: The opinions expressed in This Week In Education are strictly those of the author and do not reflect the opinions or endorsement of Scholastic, Inc.