1. I'm pretty sure that the metric
they used this time around--10% of income or $5,000 would still make
Patty Barreiro's bankruptcy a "medical bankruptcy", something which has
never been asserted by anyone. The proximate cause of that bankruptcy
was (according to Andrews) a lawsuit and an income crunch; the medical
bills were simply added in.
2. The response rate on their survey
was only 20%. Given the deep shame surrounding bankruptcy, you have to
worry that they got an unrepresentative sample. And how is that sample
most likely to be unrepresentative? Well, one pretty likely way is
that people who went bankrupt through no fault of their own--folks who
got whacked by large and unpayable medical bills or a business
closure--were more likely to respond than the people with drug or
alcohol problems, profound depression that left them unable to work,
compulsive gambling issues, and so forth.
3. The authors have
an odd tendency to ignore what the respondents themselves say. 32% of
those surveyed about their 2007 bankruptcies--not 62%--reported that
"medical problem of self or spouse was reason for bankruptcy." If
anything, I would expect this to be overreported, since this is one of
the few reasons for bankruptcy that does not trigger shame in our
society; there should be a tendency for people to overemphasize the role
that illness played in bankruptcies, which are often multifactorial and
involve many different kinds of bills.
Warren, et al argue that
they need to correct for the fact that someone may mortgage a home to
pay for an illness, and then report the reason for their bankruptcy as
"unable to pay for mortgage". Fair enough, but if you are going to try
to correct for that sort of thing, you also need to try to correct for
bias in the other direction. They only make adjustments that ratchet
the results upwards.
More troubling, when you look at the
percentage of people who self report that illness played a role in their
bankruptcy, it hasn't changed that much. In the 2001 study, "illness
or injury" is cited causally in just over 28% of cases. In the 2007
study, it's 32%. That's about a 10% increase, far less dramatic than
they claim, and given how small their sample is, not particularly
compelling. This should have induced far more caution in describing the
changes than they claim.
4. Their methodology is quite
explicitly designed to capture
every case where medical bills, or medical loss of income, coexist with
some other causal factor--but the medical issues are then always
designated as causal in their discussion. As the case of Patty Barreiro
indicates, this is simply not correct. Allow me to channel one of last
First, she redefines medical-related
bankruptcy as having debts of
$5000 or 10% of family income. She packages this as conservative, but
for dual-unemployed families (income zero), ANY medical debt is
therefore medical-related. She has a series of OR conditions which are
likely to inflate cases where a person has been sick but where it ISN'T
related to their bankruptcy. Then she runs logistic
regression to predict medical bankruptcy (at least as she defines it).
It's not totally clear to me what variables she used or the order they
were entered-- forward stepwise basically leaves it to the computer. All
those other predictors that were skipped need to be declared. There are
tricks with collinearity you can pull with stepwise (or, really any
regression) to inflate or deflate coefficients. I also don't see a test
Most importantly, though, we don't
see the questionnaire she used. I don't see any checks for common
methods bias, and not knowing her operationalizations, we have to
basically take them on faith. She also doesn't establish comparability
with her 2001 study, which is the whole point of her paper.
flaws like this, I'm surprised that the study made it through peer
review. I don't see a fatal flaw, but the reviewers would be sure to
demand these kinds of methodological checks, and at six pages, there's
plenty of room for that last half-page.
She seems to
define medical-related as "bankruptcies which occur due to medical
conditions, have an impact on medical conditions, or occur adjacent to
medical conditions". If I calculate "medical-related sunspots", would I
get a similar high percentage of the population? This limits its utility
for anything other than a stat that can be pasted out of context into a
flyer or campaign mailing.
. . .
I should point out that by collecting three very different kinds of
"medical-related" bankruptcy into one predictor, she makes it totally
meaningless. People teetering on the edge of bankruptcy have ALL sources
of debt very high. Delinquent housing debts, credit card debts, mall
boutique debts, and utility bills. That's why they're filing for
bankruptcy. You're supposed to use controls and good
operationalizations to eliminate those problems. That's why we use
regression in the first place. But she doesn't use it that way, in fact
she goes out of her way to avoid using it; and where she does use it,
she employs stepwise regression, which explicitly doesn't help with this
problem. We also don't know her questionnaire because she didn't
include it, even in an appendix.
So the real litmus test
for this is: if I go back to her sample and use her methodology to find
the proportion of credit card debt-related bankruptcy, I would probably
ALSO get 80% or 90%. Then, I could go back again and show that
automobile-related bankruptcy is another 40%-60% of bankruptcies. When
you strip out the statistical obfuscation, you end up with a couple
simple correlations between year and another variable that can best be
described as "we don't know what this is".
* I've actually seen
reports of a few cases where people declare bankruptcy over a couple
thousand worth of debt, and I don't understand why the lawyers who filed
those cases aren't in jail. You don't file bankruptcy over a debt that
can be cleared by working night shifts at Pizza Hut for a month--if
you're too infirm for a part time job, you probably don't have a salary
they can garnish or assets too large to hide under your mattress.