Elizabeth Warren and the Terrible, Horrible, No Good, Very Bad, Utterly Misleading Bankruptcy Study

Elizabeth Warren has another study out showing that medical expenses contribute to more than half of all bankruptcies--indeed, this time, it's 70%, up from the 50% she found in 2001. 

Now, it is possible that this is true.  The fact that it seems to disagree with every other study I've ever read that is not authored by Elizabeth Warren, and also, the self-reports of the people in her study (only about a third of whom attribute their bankruptcy to a health problem) could just be a fluke.  It doesn't necessarily mean that it's wrong.  

Yet upon closer examination, it turns out that it is not just wrong, but actively, aggressively wrong.  Warren and her co-authors have obscured important and obvious facts that call the integrity of the work into serious question.

The text itself raises a huge red flags.  It's hard to believe that more than half of people who have been pushed into bankruptcy by a medical issue don't understand this fact.  Perhaps they are not the brightest bulbs on the Christmas tree, but could it really be true that most people catapaulted into a financial crisis by their medical bills don't even notice that health care expenses are their main problem?

My radar is further engaged by the fact that they're implying a really astonishing surge in medical-bill-driven bankruptcies, in a healthcare environment that just didn't change all that massively.  Their study opens:

As recently as 1981, only 8% of families filing for bankruptcy did so in the aftermath of a serious medical problem.  By contrast, our 2001 study in 5 states found that illness or medical bills contributed to about half of bankruptcies.

Since then, the number of un- and underinsured Americans have grown, health costs have increased, and Congress tightened the bankruptcy laws.

In those six years, the percentage of uninsured families ground upward, and health care cost continued to rise at about twice the rate of inflation.  But a 2.5% real annual increase in the cost of a budget item  that accounts for something like 5% of annual household expenditures shouldn't make the bankruptcy stats jump that much. 

Perhaps there was a big increase in the volatility of those expenditures, with the average growing slowly, but a larger number of people being hit by truly massive bills?  Perhaps, but I'm aware of no data that show it.  Yes, people complain about deductible increases and more cost-shifting, but a $500 or $1000 increase in the annual deductible won't tip any family into bankruptcy, and the complaints about denied claims go back long before 2001.   For this to be causing such a huge surge in bankruptcies, health care companies would have had to discover some extraordinarily clever new way to deny people health care benefits without being sued, or fired by the companies who buy their insurance.  If they have, it hasn't been a prominent feature in the recent health-care debate.  As far as I know, they're still using the same old strategy of outlasting and/or confusing their patients.

Yet Warren, et al. claim their current results both show a dramatic increase, and are in robust agreement with their earlier study.  How could steadily, moderately rising medical bills, a roughly static business and legislative environment, and a small increase in the uninsured, possibly have driven up bankruptcies so massively?

Answer:  they didn't.  What Warren et. al. neglect to mention is that bankruptcies fell between 2001 and 2007.  In fact, they were cut in half.  Going by the numbers Warren et. al. provide, medical bankruptcies actually fell by almost 220,000 between 2001 and 2007, a fact that they not only fail to mention, but deliberately obscure.

Are Warren, et. al. unaware that bankruptcies fell by half?  No bankruptcy analyst could possibly be unaware of this fact; it has been the most talked-about phenomenon in the bankruptcy area since the 2005 law was passed.  Moreover,  they're clearly familiar with the filings data, because they use it to make their point:

The number of filings spiked in mid-2005 in anticipation of the new law, then plummeted.  Since hten, filings have increased each quarter.  They are likely to exceed one million households in 2008, representing about 2.7 million people.

What's left out here?  That in 2001, 1.45 million households filed for bankruptcy.  In 2007, that number was 727,167.   Had their paper done the basic arithmetic, readers would easily have seen that their own numbers imply a decrease in medical bankruptcies, from about 750,000 to slightly over 500,000.  Yet their paper does not merely ignore this fact; it uses language that seems deliberately designed to conceal it.  I invite any of my readers to scan the paper for any hint that medical bankruptcies had fallen significantly over 6 years.

This is elementary social science.  A huge change in the composition of your sample needs to be noted.  It certainly should not be artfully disguised.  If the 2005 bankruptcy form made it more difficult to file bankruptcy, the people who still file bankruptcy will largely be those who are forced to it by events totally beyond their control.  Medical bankruptcies seem to fill that bill. 

Yet even so, their own work shows medical bankruptcies falling in the years between 2001 and 2007, which would seem to invalidate, not support, the claim that half of all bankruptcies in 2001 were driven by medical events beyond the household's control.

Elementary googling reveals that the two doctors who co-authored this study are prominent spokespeople for Physicians for a National Health Program, and thus have an obvious agenda, one that Elizabeth Warren has not been shy about sharing.  The American Journal of Medicine, which published this study, seems to have flunked Peer Review 101--I sure hope they're more careful about controlling for background conditions when they're talking about cures for cancer  Also wearing duncecaps are the journalists who are already uncritically parroting it.

There is, of course, a large amount of terrible advocacy masquerading of social science out there, and too many journals and journalists abet it.  But this is particularly troubling because Elizabeth Warren is now in charge of overseeing the TARP program for Congress.  What other inconvenient facts is she shielding us from?