Iraq benchmarks: false precision?

A lot of liberal bloggers are understandably skeptical of this Michael O'Hanlon piece on establishing political benchmarks in Iraq.

The most intriguing area of late is the sphere of politics. To track progress, we have established “Brookings benchmarks” — a set of goals on the political front similar to the broader benchmarks set for Baghdad by Congress last year. Our 11 benchmarks include establishing provincial election laws, reaching an oil-revenue sharing accord, enacting pension and amnesty laws, passing annual federal budgets, hiring Sunni volunteers into the security forces, holding a fair referendum on the disputed northern oil city of Kirkuk, and purging extremists from government ministries and security forces.

At the moment, we give the Iraqis a score of 5 out of 11 (our system allows a score of 0, 0.5, or 1 for each category, and is dynamic, meaning we can subtract points for backsliding). It is far too soon to predict that Iraq is headed for stability or sectarian reconciliation. But it is also clear that those who assert that its politics are totally broken have not kept up with the news.

Spencer is probably the most pungent, Ezra's the soberest, and Matt wins the prize for best metaphor:

I think Brookings Benchmarks are kind of like Disney Dollars, i.e. funny money. We get no sense of where this five out of eleven comes from or what it's really supposed to signify. The general thrust of the exercise seems to be to cast "failure" as such an extreme scenario that it can never actually happen. O'Hanlon will always be wisely positioned between the over-optimists and the over-pessimists, always urging us to hang on for a couple more Friedman Units, and so the war will continue, forever and ever just as John McCain wants.

At least, I assume that it is a good metaphor. I have never actually been to Disneyworld, because my parents do not love me.

This is the core complaint: though the index numbers have a solid, factual ring about them, in fact they're quite subjective. I don't think that this actually stands very well as a general critique. We have a lot of these sort of indices--Freedom House's world freedom index, Transparency International's corruption indices, Heritage's index of economic freedom--and while they're obviously inherently somewhat subjective, they're an extremely useful tool for gauging relative performance. There may be some quibble about whether a country is a 2.4 or a 2.6 on the world freedom index, but there's a clear and substantial difference between a 2.4 and a 4.8.

But of course, those indexes depend a lot on the credibility of the source, and there's not question that Michael O'Hanlon is on the highly optimistic side of Iraq analysts. That's going to make it hard to get consensus acceptance of his figures, especially when he hasn't published what the benchmarks are, or what constitutes the difference between a 0, a 0.5, and a 1.

I follow the Iraq index pretty closely, and from what I can tell, things have unambiguously gotten better in Iraq; civilian fatalities at their lowest level ever, the police and army better staffed, and key economic indicators such as oil and electricity production slowly but fairly steadily heading upwards. Electricity is now meeting or exceeding pre-war production, and with the attacks on the infrastructure basically halted, what problems there are now center around updating Iraq's aged infrastructure, rather than rebuilding destroyed equipment. Likewise, oil is essentially at prewar levels, and without the attacks on the pipelines and installations, there is reason to believe that it may actually exceed prewar levels in the reasonably near future. There is little data on other infrastructure, education, or healthcare, which are critical numbers to have. But the numbers we do have are all pretty much moving in the right direction. So off the cuff, O'Hanlon's assertions don't seem totally crazy to me.

On the other hand, the real question is the future: can Iraq get a government that can keep things going in the right direction? That's what O'Hanlon's benchmark is supposed to measure, and a 5 isn't a particularly hopeful number. Worse, looking at the political benchmarks in the Iraq index does suggest that it may be a trifle overoptimistic.