The Limits of Policy Analysis

I have been reading with great interest the back-and-forth between Mark Kleiman and Jim Manzi on how much more humble we ought to be about new policy changes.  I know and like both men personally, as well as having a healthy respect for two formidable intellects, so I've greatly enjoyed the exchange.

Naturally, this has put me in mind of just how hard it is to predict policy outcomes--how easy it is to settle on some intuitively plausible outcome, without considering some harder-to-imagine countervailing force.

Consider the supply-siders.  The thing is intuitively appealling; when we get more money from working, we ought to be willing to.  And it is a mathematical truism that revenue must maximize at some point.  Why couldn't we be on the right-hand side of the Laffer Curve?

It was entirely possible that we were; unfortunately, it wasn't true.  And one of the reasons that supply-siders failed was that they were captivated by that one appealing intuition.  In economics, it's known as the "substitution effect"--as your wages go up, leisure becomes relatively more expensive relative to work, so you tend to do less of the former, more of the latter.

Unfortunately, the supply-siders missed another important effect, known as the "income effect".  Which is to say that as you get richer, you demand more of some goods, and less of others.  And one of the goods you demand more of as you get richer--a class of goods known as "superior goods"--is leisure.

Of course, some people are so driven that they will simply work until they drop in the traces.  But most people like leisure.  So say you raise the average wage by 10%.  Suddenly people are bringing home 10% more income every hour.  Now, maybe this makes them all excited so they decide to work more.  On the other hand, maybe they decide they were happy at their old income, and now they can enjoy their old income while working 9% fewer hours.  Cutting taxes could actually reduce total output.

(We will not go into the question of how much most people can control their hours--on the one hand, most people can't, very well, but on the other hand, those who can tend to be the high-earning types who pay most of your taxes.)

Which happens depends on which effect is stronger.  In practice, apparently neither was strong enough to thoroughly dominate, at least not when combined with employers who still demanded 40 hour weeks.  You do probably get a modest boost to GDP from tax cuts.  But you also get falling tax revenue.

Naturally, even-handedness demands that I here expose the wrong-headedness of some liberal scheme.  And as it happens, I have one all ready in the oven here:  the chimera of reducing emergency room use.  The argument that health care reform could somehow at least partially pay for itself by keeping people from using the emergency room was always dubious.  As I, and others argued, there's not actually that much evidence that people use the emergency room because they are uninsured--rather than because they have to work during normal business hours, are poor planners, or are afraid that immigration may somehow find them at a free clinic.

Moreover, we argued, non-emergent visits to the emergency room mostly use the spare capacity of trauma doctors; the average cost may be hundreds of dollars, but the marginal cost of slotting ear infections in when you don't happen to have a sucking chest wound, is probably pretty minimal.

But even I was not skeptical enough to predict what actually happened in Massachusetts, which is that emergency room usage went up after they implemented health care reform.

In retrospect, it's obvious.  Massachusetts increased access to primary care physicians, but it didn't actually increase the number of primary care physicians.  This led to wait times, which meant that some people who used to avoid the emergency room by going to see their primary care physician now ended up in the emergency room.  Meanwhile, many of the people who were using the emergency room for primary care still are, because MassCare doesn't pay them for the hours they lost by taking off work.

At the same time, MassCare suddenly lowered the marginal cost of emergency room care for many people who had previously needed to think hard before they showed up at the triage desk.  Now they were only on the hook for a smallish co-pay--not nothing, but not the hundreds of dollars that a visit would have run in the past.  In retrospect, it's not surprising at all that emergency room usage went up.

But in prospect, I didn't see it.  And I'm not aware of anyone else who saw this coming either--there may have been one or two especially perspicacious souls, but most of us were totally blindsided.

It's a good reminder to be humble about what we say about policy.  Even if you're trying to be as fearlessly honest with yourself as possible, it is almost certain that you are overweighting some intuitions, and missing other forces entirely.

Does that mean that we should never do anything?  Well, I'll leave the debate about how much such humility should curtail our actions for another day.  Right now, I'll settle for noting that it should curtail the terms of debate.  Think just how certain most of the predictions were, in both these cases. And in both these cases, the people advocating for a given action based on their certitude were simply 100% wrong.  Not because conservatives, or liberals, are espeically stupid or ignorant.  But just because the universe is so complex that it can almost always find a way to surprise us.