Forget Excel: This Was Reinhart and Rogoff's Biggest Mistake

Correlation is not causation

Harvard professor Ken Rogoff has not had a good week (Reuters)

For an economist, the five most terrifying words in the English language are: I can't replicate your results. But for economists Carmen Reinhart and Ken Rogoff of Harvard, there are seven even more terrifying ones: I think you made an Excel error.

Listen, mistakes happen. Especially with Excel. But hopefully they don't happen in papers that provide the intellectual edifice for an economic experiment -- austerity -- that has kept millions out of work. Well, too late. As Mike Konczal of the Roosevelt Institute reported, Thomas Herndon, Michael Ash, and Robert Pollin of the University of Massachusetts, Amherst, have found serious problems with Reinhart and Rogoff's austerity-justifying work. That work, which shows that countries with public debt of 90 percent of GDP or more tend to grow slower, omitted data for five of its 19 countries, and used the wrong data for another. The former was, embarrassingly enough, due to an Excel misadventure, and the latter an unrelated issue. If you use all of the (right) numbers, it turns out growth does slow when debt is high, but not nearly as much as Reinhart and Rogoff -- hereafter, R-R -- claimed.

In other words, there is no evidence for anything resembling a growth tipping point when debt hits 90 percent of GDP.

This is the academic's version of the dream where you're naked in public. Except it's not a dream. It's the mortifying reality for R-R, who have admitted that they forgot to drag their Excel formula down five more cells. But it's worse than mortifying for everybody else. It's been a catastrophe. Not that R-R made a pretty galling mistake; rather, that such a flawed paper gave the intellectual ballast to an idea that has failed everywhere it's been tried the past few years. Now, policymakers would have pursued austerity regardless, but R-R gave them a reason (and seemingly a bright red 90 percent of GDP line) to do so sooner. If too much debt is associated with too little growth, then there's no time to lose for slashing deficits.

Those are important words: "associated with". As I pointed out before, the best argument against taking R-R as austerity's gospel truth was it was just a correlation. Of course a ratio tends to increase more when its denominator increases less. That's how fractions work. But it doesn't prove that the rising ratio causes the stagnating denominator. If anything, the causality runs the other way -- lower growth tends to cause higher debt, as tax revenue falls and safety-net spending rises during a slump. Indeed, as you can see below, Arindrajit Dube of the University of Massachusetts, Amherst, found that debt-to-GDP predicts past GDP growth much better than future GDP growth. In other words, higher debt doesn't cause lower growth as much as lower growth causes higher debt.


Of course, this hasn't stopped deficit hawks from touting R-R's work as proof that we must tackle the long-term debt and we must tackle it now. Including, sometimes, R-R themselves. Now, in their paper, R-R are careful to say the relationship between higher debt and lower growth is just that: a relationship. In fact, that's been their defense: they never said it was anything more than a correlation, and that correlation still holds after correcting for all their mistakes, albeit not as strongly.

That's true ... if you only look at what they said in their paper, and ignore what they said about their paper. For example, here's what they said in Bloomberg View back in July 2011:

Our empirical research on the history of financial crises and the relationship between growth and public liabilities supports the view that current debt trajectories are a risk to long-term growth and stability, with many advanced economies already reaching or exceeding the important marker of 90 percent of GDP.... 

The biggest risk is that debt will accumulate until the overhang weighs on growth.... 

Those who remain unconvinced that rising debt levels pose a risk to growth should ask themselves why, historically, levels of debt of more than 90 percent of GDP are relatively rare and those exceeding 120 percent are extremely rare (see attached chart 2 for U.S. public debt since 1790). Is it because generations of politicians failed to realize that they could have kept spending without risk? Or, more likely, is it because at some point, even advanced economies hit a ceiling where the pressure of rising borrowing costs forces policy makers to increase tax rates and cut government spending, sometimes precipitously, and sometimes in conjunction with inflation and financial repression (which is also a tax)?

To be fair, R-R do say that they only found that higher debt and lower growth are "associated" and that there's no "bright red line" (even if policymakers interpret it that way) at 90 percent. But they also make it quite clear that they think their correlation is more than just a correlation. They think higher debt causes lower growth, and, after a little throat-clearing, they're not too shy about saying so.

This wasn't the only time they dressed up their correlation as causation. Here's a piece Ken Rogoff wrote for Project Syndicate in June 2012:

In a series of academic papers with Carmen Reinhart - including, most recently, joint work with Vincent Reinhart ("Debt Overhangs: Past and Present") - we find that very high debt levels of 90% of GDP are a long-term secular drag on economic growth that often lasts for two decades or more.... 

Of course, there is two-way feedback between debt and growth, but normal recessions last only a year and cannot explain a two-decade period of malaise. The drag on growth is more likely to come from the eventual need for the government to raise taxes, as well as from lower investment spending. So, yes, government spending provides a short-term boost, but there is a trade-off with long-run secular decline.

It's the same pattern: a few caveats, and then a semi-speculative overselling of their results. But their biggest overselling didn't come in the media. It came behind closed doors -- in Congress. Tim Fernholz of Quartz flagged the following passage from Senator Tom Coburn's recent book about the time R-R briefed members of Congress in April 2011, a few months before the debt ceiling debacle:

Johnny Isakson, a Republican from Georgia and always a gentleman, stood up to ask his question: "Do we need to act this year? Is it better to act quickly?" 

"Absolutely," Rogoff said. "Not acting moves the risk closer," he explained, because every year of not acting adds another year of debt accumulation. "You have very few levers at this point," he warned us. 

Reinhart echoed Conrad's point and explained that countries rarely pass the 90 percent debt-to-GDP tipping point precisely because it is dangerous to let that much debt accumulate. She said, "If it is not risky to hit the 90 percent threshold, we would expect a higher incidence."

R-R whisper "correlation" to other economists, but say "causation" to everyone else. Now, they don't always say it outright -- at least not at first. Rather, they say "this isn't definitely causation ... but come on, what else could it be?" That's been more than enough for the austerians who have been desperate for any kind of justification to forget about unemployment and worry about debt instead.

The boring reality is the relationship between public debt and growth isn't clear. As Justin Fox of Harvard Business Review points out, there simply isn't enough data. Some countries run up big debts fighting wars, and then grow fine. Some countries run up big debts fighting financial crises, and then grow slowly as the private sector deleverages. Some countries run up big debts as a matter of course, and then grow slowly as rising rates crowd out private investment. And even the few data points we do have don't always tell us all that much. Indeed, as Paul Krugman points out, it shouldn't surprise us that the U.S. has averaged negative growth during its high debt years, because most of those years came during the World War II demobilization. In other words, it's impossible to say anything dispositive about debt and growth more broadly.

But that hasn't stopped R-R from trying. This kind of overhyping is why Joe Weisenthal called them "the most dangerous economists in the world" back in 2011. And it's a far more damning error than anything they did with Excel.