About this blog Subscribe to this blog

Reform: Value-Added Ratings -- For Think Tanks

Picture 4Everyone gets rated and ranked these days -- teachers, schools, districts, states -- some of these ratings are quite elaborate.  But not think tanks, or at least not education think tanks.  

This seems interesting and unfair to me -- and an opportunity for someone to step in make a name for herself.  After all, just this week AEI's Rick Hess named his top edu-scholar (Linda Darling-Hammond) of the year, and the Washington Post's Ezra Klein named its all-purpose think tank of the year (Center on Budget and Policy Priorities).  

Some of the key measures would have to be original research, media mentions, and political influence. Research quality.  Cost-per-report.  Some of the obvious contenders would be Fordham, Education Sector, AEI, CAP, New America, the Education Trust.  Heritage? The Alliance?  EPI?  I'm not sure.  I'm not even sure I could say what constitutes a think tank these days, given how many have sprawled out into space once occupied by newspaper columnists (the "flipped" think tank).  Maybe some Gates Foundation program officer has already created such a ratings system and I just haven't heard about it yet.

I know and like people at all of these places but in recent months, I've found the detailed New America blog postings to be the most interesting and informative.  And I remain stubbornly convinced that the Education Trust is the most influential in terms of real-world impact though it lacks a (c)4 Action Fund like CAP and I haven't seen a report from them in what seems like years. 


Feed You can follow this conversation by subscribing to the comment feed for this post.

I wish people would stop calling these companies "think tanks" and start calling them "advocacy groups," because their primary purpose is advocacy, not thinking.

Tim raises an important problem for rating these entities. We have to rate the value that is added by their advocacy. Your preferences about political influences or research quality are irrelevant; is the entity is able add value to standardized test scores?

Value Added Ratings are data-driven. We would have to identify the specific students affected by each group's advocacy, and then administer high-stakes tests to those students. Wireless generation would then design and apply a proprietary metric which compared the students' growth on the standardized tests with their expected growth, based on the performance of a matched cohort a of non-affected students.

Chicago is a good place to start. After the past decade of dazzling success there, the corporate reform think tank advocates must have bushels of value-added to total up.

Brilliant. Why didn't I think of this. VAM for every edu-wonk and higher ups in the ed-sector.

Nice post.. I think it's only a matter of time this happens. Not a matter of if, but when and by whom.

I like your categories, too..

Original Research. I would code this in terms of data collection (primary vs. secondary vs. aggregating others).. that should address this organizational spectrum think tank - advocacy group. At least to some degree.

Media Mentions. This can be tricky because of the web. But I think certain media would have to be weighted based on their total reach - print circulation and/or online visits/month.. data collection on latter would be difficult and a challenge for reliability.

Political Influence. Another tough measure for definition and data collection. But I would separate national vs. state.

Research Quality. I think this could be broken out into a few or more categories, without having to pass judgment on individual reports. Could address issues like (1) type of data collection--see above, (2) transparency of methods, (3) quality of citations or sources.. off the top of my head.

Cost Per Report. Without doing a survey of think tanks/groups, probably no way to get at this measure... and even if a survey is done, I'm not sure an organization will give information necessary to come up with the number.

I would add at least one more category...

Relationships (or "Social Capital").. Take an organization's report or observe an event advertisement or press release, how many other organizations are listed as sponsors or partners? This could be a crude measure for social capital, but I think one that could complement the Political Influence measure.

Hopefully, someone does this in the near future. Like you say, it would be a valuable annual evaluation. A potentially helpful signal (and filter) for media and policymakers.

Keep up the good work.

Ick. Unfortunately, there's no way to rate whether they are a net positive or a net negative to the world. (Or, to be more simplistic, whether they do good or evil.)

Rating them on media mentions would be depressing. The so-called "think tanks" (yes, this term is ENTIRELY inaccurate and misleading) have the mainstream media eating out of their hands. Here in my hometown, there was a long period where the San Francisco Chronicle would have printed the Hoover Institution's grocery list, while haughtily disdaining heartfelt commentaries submitted by actual advocates in the community. That has improved a tiny bit as the Chronicle is desperate for ANY attention from the public now, but only a bit.

The comments to this entry are closed.

Disclaimer: The opinions expressed in This Week In Education are strictly those of the author and do not reflect the opinions or endorsement of Scholastic, Inc.