Susan Feinberg of Rutgers Business School writes:
I'm a professor at Rutgers and an economist. After reading your piece, I had a look at the article and a few things came out clearly.
First, it's somewhat shocking that the authors in the paper tolerated a nearly 50% attrition in the sample in conducting phone interviews. 906 out of 2314 people who filled out surveys couldn't be contacted? There is a huge selection problem here. One can think of all kinds of reasons why people who were contactable might disproportionately suffer from medical conditions. Many of the 906 "disappeared" people doubtless moved on - physically - because they *could*. One could speculate ad infinitum on systematic differences between the 1032 interviewees and the 906 "disappeared." But the problem is, these 906 are lost, and it would idle speculation.
Accepted practice, particularly for scholars with resources (!), would be to try to track down the 906 people and, at least, attempt to obtain some information on them so that the final econometric model would be a Heckman-type selection model which estimates the joint probability that a person would be in the phone interview sample along with the bankruptcy prediction model.
This is the main issue I see, but there are many others. I find the use of means, rather than medians very odd. The median medical bill is probably zero, or close to zero, and, as you point out, the means are likely biased up by a small group of large numbers. There would have been no need to "trim outliers at 100K" if medians, rather than means, were used to describe the sample.
I am also bothered by the use of "debtor or spouse lost 2+ weeks of income due to illness or disability." Again, what we don't observe here is individual heterogeneity. This is the issue with regard to the selection problem discussed above. It's probably true that people who file for bankruptcy are at the margin of society for all kinds of reasons. They probably have more stress and financial concerns generally, miss more days off work for reasons like car trouble and childcare (which they count as "sick days"), etc. If the individuals called the time off "sick days" with regard to their employers, it's not surprising that such a high proportion of people - 38+% fall into the 2+ week category. I don't think a positive answer to this question should qualify as a "medical" bankruptcy.
Another strange entry in Table 2 is "Medical Bill Problems" which are people who have positive answers to any of the first three questions. We can infer from the fact that 57% of respondents have "medical bill problems" but only 29-35% of people say their bankruptcy was due to medical bills OR report having high medical bills that there is very little overlap between the first and second question. I would expect an almost perfect mapping of question 1 (self-report that medical bills caused bankruptcy) and question 2 (self-reported large medical bills). And yet, there seems to be almost no overlap, since 57% of people answered yes to EITHER question 1 or question 2, but only about 30% answered yes to each individual question. I'm ignoring question 3 because of the small numbers.
Finally, there are all the obvious problems with self-report data, and the fact that this is a very sensitive topic. Getting "true" self-report data on reasons for bankruptcy is like trying to get "true" self-report data about sex. I hope the authors consulted a reputable psychometrician to help them design their survey instrument!
The only point you make that I would quibble with regards the absolute drop in number of medial bankruptcies. Even if we take the paper's estimates of 500K to be correct, this represents a large increase in medical bankruptcies as a percent of total bankruptcies. This is an important number that, if correct, should not be ignored. It's the weakest point in your critique.
Thanks again for your thoughtful piece. I would have posted this as a comment if I could have figured out how to do it! I strongly believe in a national health insurance option, but I feel that my cause is poorly represented by bad science.
I should make it clear as I didn't in my earlier posts that I think the proportion is relevant, and indeed points to a possibility that would tell us a lot about medical bankruptcies: that they're much "stickier" than other sorts of bankruptcies. They may be harder to time, or they may simply be less sensitive to the various costs of filing for discharge. Indeed, I think that the authors' drive to prove that medical bankruptcies were being increased due to pressure from steadily higher medical bills obscured this potentialy quite important finding.
But I think that without knowing that the absolute number fell, the proportion by itself is misleading. The reverse is also true. If I had reason to believe that the absolute number of medical bankruptcies had fallen, I would never write about it without also pointing out that bankruptcy overall had fallen much more sharply than medical bankruptcies had. If I omitted that latter fact, my article would be grotesquely misleading. If I had omitted this fact, and instead filled my article with descriptions of S-Chip expansion, the growing proportion of the population eligible for Medicare, and a downtick in immigration (immigrants disproportionately tend to be uninsured), I would deserve all the righteous hell I would no doubt receive from the liberal blogosphere.
We want to hear what you think about this article. Submit a letter to the editor or write to email@example.com.