Considering Elizabeth Warren, the Scholar

(Part one of a two-part series)

The progressives seem to have made Elizabeth Warren their cause-du-jour.  I have a long and complicated history with Elizabeth Warren, so allow me a moment to offer my long and complicated thoughts on her.  Really long.  So long that I had to break it into two parts--scholarship and public life--in order to prevent the nausea, daytime sleepyness, and intracranial bleeding that might otherwise result.  Consider yourselves warned.

I first encountered Elizabeth Warren in the early part of the decade, when I read her book, The Two Income Trap.  The thesis is innovative and, I think, at least partly correct:  that in many ways, two income families have made households less financially stable, not more so.  Among higher income families, much of the extra income has simply been poured into a bidding war with other higher income families for homes in good school districts.  Among lower income families, much of the extra income has gone to replacing home production with market production:  convenience meals, work clothes, second cars, child care, and so forth.

Because the extra income is being fully consumed, the result is that if one partner loses their job, the family is not more insulated from economic shock, but less so.  In the 1950s, if Dad lost his job, Mom could pick up extra work to make up at least some of the loss.  In the Mean Teens, Mom's already got a full time job.

But while I found the thesis compelling, there were some problems with the book.  The first is that Warren simply fails to grapple with what her thesis suggests about the net benefits of the two-earner family.  Admittedly, I don't quite know what to say either, but at least I can acknowledge that it's a pretty powerful problem for the current family model; Warren kind of waves her hands and mumbles about social programs and more supportive work environments.  There is no possible solution outside of a more left-wing government.

But the deeper problem is that some of her evidence doesn't really support her thesis, and can be made to appear to support her thesis only by making some very weird choices about what metrics to use.

There are a lot of small points (and it's been a long time since I read the book), but I'll take just one, which I blogged some years ago.  In order to show that the extra income from two income families is going to bidding up the prices of housing (rather than, say, buying better housing), Warren argues that housing consumption hasn't actually increased much in the last few decades:  by less than a room per house.

But since the starting size was five rooms per house, that's actually a rather large increase:  twenty percent more rooms.  Moreover, at the same time that housing sizes were rising, divorce, later marriage, and fewer children were causing the average household size to drop rather precipitously, from about 3.3 to about 2.6 persons.  As I noted in that long-ago blog post, that means that the number of rooms per household has gone up by nearly a whole room per person.  That's actually quite a lot of extra housing consumption.

Moreover, we know that the square footage of new homes has increased dramatically since 1960--nearly doubled, by most estimates I've seen.  That even as the number of people has fallen.  Again, this is a lot of extra housing consumption.

These are obvious issues she should have dealt with.  (I'll add that Ed Glaeser's work indicates a strong role for zoning and other sorts of NIMBYism, but it wasn't particularly famous at the time, so there's no reason she should have known about it.)  But they considerably weaken her thesis, and she doesn't have a good answer for them.

That's a pattern I see over and over in her work.  In her (in)famous paper on medical bankruptcies in 2001, Warren and her co-authors defined anyone with $1000 worth of medical bills as having a medical bankruptcy, and used that figure to imply that rising medical bills were pushing people over the financial edge.  Now maybe they are, but you sure couldn't prove it with that metric.  I hope to hell that no lawyer (Warren is a law professor) would advise a client with no debt but $1,000 worth of medical bills to declare bankruptcy, because doing so would be malpractice*.

If $1,000 worth of medical bills can push you into bankruptcy, you already had a major problem, either on the spending side or on the income side.  There is simply no reason for using this metric as a proxy for anything.  As a side note, this would make the bankruptcy of Edmund Andrews' wife a "medical bankruptcy", since she had thousands in bills to some sort of cosmetic practice (I believe a dermatologist).

I blogged about her latest paper here.  This one also looked at medical bankruptcies, this time in 2007, and was supposed to improve on the problems of the previous paper.  These were the conclusions they came to:

In 2007, before the current economic downturn, an American family filed for bankruptcy in the aftermath of illness every 90 seconds; three quarters of them were insured.

Since 2001, the proportion of all bankruptcies attributable to medical problems has increased by 50%.  Nearly two thirds of all bankruptcies are now linked to an illness.

How did medical problems propel so many middle-class, insured Americans toward bankruptcy?  For 92% of the medically bankrupt, high medical bills directly contributed to their bankruptcy.  Many families with continuous coverage found themselves under-insured, responsible for thousands of dollars in out-of-pocket costs.  Others had private coverage but lost it when they became too sick to work.  Nationally, a quarter of firms cancel coverage immediately when an employee suffers a disabling illness; another quarter do so within a year.  Income loss due to illness also was common, but nearly always coupled with high medical bills.

The present study and our 2001 analysis provide the only data on large cohorts of bankruptcy filers derived from in-depth surveys.  As with any survey, we depend on respondents candor.  However, we also had independent checks--from court records filed under penalty of perjury--on many responses. Because questionnaires and court records were available for our entire sample, we used them for most calculations.  The lowest plausible estimate of the medical bankruptcy rate from those sources is 44.4%--the proportion who directly said that either illness or medical bills were a reason for bankruptcy.  But many others gave reasons such as "aggressive collection efforts" or "lost income due to illness and had large medical debts.  Indeed, detailed telephone interview data available for 1032 debtors revealed an even higher rate of medical bankruptcy than our 62.1% estimate--at least 68.8% of all filers.

. . . Teasing causation from cross-sectional data is challenging.  Multiple factors push families into bankruptcy. Yet, our data clearly establish that illness and medical bills play an important role in a large and growing proportion of bankruptcies.

This paper was deeply, deeply flawed, and all in ways that suggested--as this discussion does--that rising medical bills were causing an increase in bankruptcies.  I'll try to hit the highlights:

1.  I'm pretty sure that the metric they used this time around--10% of income or $5,000 would still make Patty Barreiro's bankruptcy a "medical bankruptcy", something which has never been asserted by anyone.  The proximate cause of that bankruptcy was (according to Andrews) a lawsuit and an income crunch; the medical bills were simply added in.

2.  The response rate on their survey was only 20%.  Given the deep shame surrounding bankruptcy, you have to worry that they got an unrepresentative sample.  And how is that sample most likely to be unrepresentative?  Well, one pretty likely way is that people who went bankrupt through no fault of their own--folks who got whacked by large and unpayable medical bills or a business closure--were more likely to respond than the people with drug or alcohol problems, profound depression that left them unable to work, compulsive gambling issues, and so forth.

3.   The authors have an odd tendency to ignore what the respondents themselves say.  32% of those surveyed about their 2007 bankruptcies--not 62%--reported that "medical problem of self or spouse was reason for bankruptcy."  If anything, I would expect this to be overreported, since this is one of the few reasons for bankruptcy that does not trigger shame in our society; there should be a tendency for people to overemphasize the role that illness played in bankruptcies, which are often multifactorial and involve many different kinds of bills.

Warren, et al argue that they need to correct for the fact that someone may mortgage a home to pay for an illness, and then report the reason for their bankruptcy as "unable to pay for mortgage".  Fair enough, but if you are going to try to correct for that sort of thing, you also need to try to correct for bias in the other direction.  They only make adjustments that ratchet the results upwards.

More troubling, when you look at the percentage of people who self report that illness played a role in their bankruptcy, it hasn't changed that much.  In the 2001 study, "illness or injury" is cited causally in just over 28% of cases.  In the 2007 study, it's 32%.  That's about a 10% increase, far less dramatic than they claim, and given how small their sample is, not particularly compelling.  This should have induced far more caution in describing the changes than they claim.

4.  Their methodology is quite explicitly designed to capture every case where medical bills, or medical loss of income, coexist with some other causal factor--but the medical issues are then always designated as causal in their discussion.  As the case of Patty Barreiro indicates, this is simply not correct.  Allow me to channel one of last year's commenters:

First, she redefines medical-related bankruptcy as having debts of $5000 or 10% of family income. She packages this as conservative, but for dual-unemployed families (income zero), ANY medical debt is therefore medical-related. She has a series of OR conditions which are likely to inflate cases where a person has been sick but where it ISN'T related to their bankruptcy.

Then she runs logistic regression to predict medical bankruptcy (at least as she defines it). It's not totally clear to me what variables she used or the order they were entered-- forward stepwise basically leaves it to the computer. All those other predictors that were skipped need to be declared. There are tricks with collinearity you can pull with stepwise (or, really any regression) to inflate or deflate coefficients. I also don't see a test for homoscedasticity.

Most importantly, though, we don't see the questionnaire she used. I don't see any checks for common methods bias, and not knowing her operationalizations, we have to basically take them on faith. She also doesn't establish comparability with her 2001 study, which is the whole point of her paper.

With flaws like this, I'm surprised that the study made it through peer review. I don't see a fatal flaw, but the reviewers would be sure to demand these kinds of methodological checks, and at six pages, there's plenty of room for that last half-page.

She seems to define medical-related as "bankruptcies which occur due to medical conditions, have an impact on medical conditions, or occur adjacent to medical conditions". If I calculate "medical-related sunspots", would I get a similar high percentage of the population? This limits its utility for anything other than a stat that can be pasted out of context into a flyer or campaign mailing.

. . .

I should point out that by collecting three very different kinds of "medical-related" bankruptcy into one predictor, she makes it totally meaningless. People teetering on the edge of bankruptcy have ALL sources of debt very high. Delinquent housing debts, credit card debts, mall boutique debts, and utility bills. That's why they're filing for bankruptcy.

You're supposed to use controls and good operationalizations to eliminate those problems. That's why we use regression in the first place. But she doesn't use it that way, in fact she goes out of her way to avoid using it; and where she does use it, she employs stepwise regression, which explicitly doesn't help with this problem. We also don't know her questionnaire because she didn't include it, even in an appendix.

So the real litmus test for this is: if I go back to her sample and use her methodology to find the proportion of credit card debt-related bankruptcy, I would probably ALSO get 80% or 90%. Then, I could go back again and show that automobile-related bankruptcy is another 40%-60% of bankruptcies.

When you strip out the statistical obfuscation, you end up with a couple simple correlations between year and another variable that can best be described as "we don't know what this is".

5.  As I discussed at the time, early 2007 is a terrible, horrible, no good, very bad time to do any sort of study on bankruptcy. In 2005, Congress passed a bankruptcy reform bill that (among other things) tightened up the procedures for filing Chapter 7, the kind of bankruptcy where you just liquidate all your debts and don't have to go on a payment plan.  This tightening was real, but rather exaggerated in the popular press (the overwhelming majority of people who need to file bankruptcy will have no problem qualifying for a Chapter 7).  Perhaps for that reason, even bankruptcy experts were stunned by the number of people who rushed to file before the six month grace period in the law was up.  They were also stunned by the magnitude of the decline, and how long bankruptcies took to start trending back towards previous levels.  This graph from Credit Slips shows what it looked like:

The study by Warren et. al. was done in early 2007, just as the rebound was gaining steam.  There's quite a lot of reason to think that the sample was badly skewed.  Many bankruptcies are long slow train-wrecks, the problems accumulating over a number of years as spending persistently outpaces income.  A pretty convincing paper argues that the single best predictor of bankruptcy is simply how much debt you've accumulated--not income, job loss, divorce, or what have you.  People who declare bankruptcy tend to have nicer stuff than others at the same income level.

But that paper also shows that medical events have independent predictive power.  And medical events are much less of a foreseeable problem.  If you're a plumber who has a stroke, you may well end up in bankruptcy simply because you lose income while you can't work (the medical bills may or may not play a large causal role).

So one worries that the people who rushed to declare bankruptcy before the 2005 change were the people with accumulating problems.  One year out, it's reasonable to be concerned that the sample will consist mostly of people who, for one reason or another, couldn't see it coming--business failures and people with health problems being the obvious two candidates.

Now you can argue that this might not have been a problem. And indeed, maybe it wasn't.  For that matter, maybe it biased the sample the other way--towards undercounting the increase since 2001.  But the question remains:  why do the study then?  Why not wait a little while?  Even another year would have substantially reduced this issue, as the people with accumulating financial problems re-entered the sample.

The authors deal with this problem in a thoroughly unconvincing way:

BAPCPA's effects appear nonselective.  Current filers differ from past ones mainly in having struggled longer with their debts⁷.  New restrictions fall equally on medical and nonmedical bankruptcies, with no preferences for medical debts or sick debtors.  It is implausible to ascribe the growing predominance of medical causes of bankruptcy to BAPCPA.

Conversely, there is ample evidence that the financial burden of illness is increasing.

The paper they cite doesn't actually say what they imply; it addresses unsecured debt and income, not medical debt specifically.  All bankruptcy is highly correlated with the level of unsecured debt, because that's the kind of debt you can most easily discharge in bankruptcy.  Its major finding is that consumers who file bankruptcy have a higher level of absolute debt than in the past--something they cite as "consistent with the view" of a lengthier struggle (as is a modest rise in the percentage of people citing that they struggled with bills for at least two years before filing).  Moreover, they specifically argue that their data challenges the thesis of the 2001 paper:

Yet the Sullivan-Warren-Westbrook model also cannot explain our findings on its own. It does not predict the sharp decline in bankruptcy filings we see in our data. Because there have been no dramatic improvements in health care delivery, employment prospects or family stability, their model would predict constant, not diminished, rates of filing.

6.  Bringing me to my next point:  the paper thoroughly obscures the point that by their own calculations, the number of medical bankruptcies fell quite dramatically between 2001 and 2007.  This is, to put it mildly, sort of a problem for the thesis that exploding medical bills are shoving people into bankruptcy.  They avoid dealing with this problem by citing only proportions, never absolute numbers--even thought they use absolute numbers freely whenever they are eye-popping. 

Of course BAPCA put downward pressure on all bankruptcies, but medical bankruptcies should be among the least amenable to prior planning.  You can try to sell the house, decide to struggle on in a bad marriage, or go to Gambler's Anonymous--but you can't decide not to have a heart attack this year.  If medical bills really are soaring as dramatically as the authors imply, we should have seen an absolute rise between 2001-2007.

7.  The authors dramatically underweight the role of income loss resulting from illness in their discussion.  They also sort of failed to mention it when talking to the media.  When I wrote my rather fiery critique last year, a lot of people said I should really criticize the media for screwing up the story.  But every interview I read of the authors, particularly Woolhandler and Himmelstein of Physicians for National Health Care, basically put all the emphasis on medical bills.  If the press got it wrong, it's because the authors went out of their way to put an extreme spin on their findings.

8.  There are a bunch of other fiddling red flags, like the fact that there's no difference in the percentage of "medical" and "non-medical" bankruptcies where the family lacks insurance at the time of bankruptcy.  They see this as evidence of underinsurance.  I  see this as frankly unbelievable, and at the very least, in need of much better explanation than they offer.  There's a slight difference in the rate of lapsed coverage at some point during the prior two years, but it's still so small as to suggest that there are a lot of "medical" bankruptcies in their sample with some other real cause.

Does this persistent tendency to choose odd metrics that inflate the case for some left wing cause matter?  If Warren worked at a think tank, you'd say, "Ah, well, that's the genre."  On the other hand, you'd also tend to regard her stuff with a rather beady eye.  It's unlikely to have been splashed across the headline of every newspaper in the United States.  Her work gets so much attention because it comes from a Harvard professor.  And this isn't Harvard caliber material--not even Harvard undergraduate.

So I think it matters on two levels. One, it matters how we evaluate the work--and I've been disappointed at how uncritically some people I really respect have been willing to accept the 2001 and 2007 findings.  Not because I don't think that there are medical bankruptcies in the United States; I do!  I think medical bills are certainly the primary cause of some of those filings, though I don't know how many, and I've also been writing about bankruptcy long enough to know that assigning any one cause to the ultimate financial meltdown is, in many cases, impossible.  (If you have nice consumer goods and no health insurance, does a car accident count as a "medical bankruptcy" or a budgeting deficiency?  If you lived right up to the edge of your income, was a job loss, or your spending pattern, to blame?  How you answer these questions depends on a large number of prior value judgments that are hotly contested in our society.)

It matters that we get this stuff right.  I am among the majority who would like to see bankruptcies reduced in this country, and we're not going to be very effective at that if we run around thinking we can cure 2/3 of them by putting a national health care system in place, when in reality a third or less have any strong causal relationship with medical bills.  Obviously, this was also held out as an argument for PPACA, making an implicit promise to the American people which I believe to be false.

But it also matters because a large part of Warren's prominence comes from the fact that she's an academic.  If she came from . . . well, the sort of think tank that publishes this sort of advocacy science . . . she would have considerably less glamor, and power.

And perhaps it mattes most of all because this woman is now under consideration to head a powerful new agency.  If this is how she evaluates data, then isn't that going to hamper her in making good policy?  If we're going to have a consumer financial protection agency, I want one that has a keen eye to the empirical evidence on consumer welfare--not one that makes progressives most happy by reinforcing their prior beliefs.

* I've actually seen reports of a few cases where people declare bankruptcy over a couple thousand worth of debt, and I don't understand why the lawyers who filed those cases aren't in jail.  You don't file bankruptcy over a debt that can be cleared by working night shifts at Pizza Hut for a month--if you're too infirm for a part time job, you probably don't have a salary they can garnish or assets too large to hide under your mattress.