Update: A test case for the media

My earlier post about Paul Krugman's attack on Rudy Giuliani's health-care ad has attracted critical comments [posted at my FT blog] that are making similar points. I argued that Giuliani's numbers were wrong and that his claims were misleading, but that cancer survival rates are indeed higher in the United States (especially for those with health insurance) than for NHS patients in England. Here is one of the comments disagreeing with that post, which states criticisms that others have also made in a fairly clear and comprehensive way.

Mr. Crook's criticisms of Paul Krugman are not even internally consistent, much less supported by fact.

1. The statement "If you are a man with prostate cancer, it may not matter very much whether your cancer is diagnosed early, or whether you live in Britain or America" implies that the survival rates are very similar.

2.  The further implication is that Mr. Crook knows the true incidence in each country, which are clouded by differences in diagnostic methodology. In reality, there may be real differences in incidence due to differences in the environment, nurture, genetic variation, and so on. Therefore, the true incidence in each country is unknown.

3.  Mr. Crook provides no citations for his assertions, while Mr. Krugman does.

4. The debate over whether PSA antigen testing affects survival is not settled, as a review of recent literature will confirm.

5. Since Mr. Crook asserts that diagnosis is much better in the United States than in the UK, it is disingenuous to shift from overall survival rates to 5-year survival rates. The Ezra Klein reference is clearly to overall mortality.

I was very disappointed by this piece, especially considering the nice promotion this site received from Brad DeLong.

Yes: thank you, Brad, for the nice promotion. Now, as to the criticisms of the post, there is some confusion over terminology here, to which I have no doubt contributed. The standard measure of the effectiveness of cancer treatment is five-year survival rates, often abbreviated in the literature to "survival rates". Whenever I said "survival rates" in my original post, I meant "five-year survival rates".

The five-year survival rate is the proportion of people still alive five years after being diagnosed with cancer. It is a flawed measure, as I tried to explain, and prostate cancer is probably the best example of how it can mislead. Early detection increases the five-year survival rate, but in the case of prostate cancer does not much reduce mortality, because prostate cancer develops so slowly that its victims tend to die of something else first. Mortality rates for that cancer are about the same in both countries. Five-year survival rates are far higher in the United States--so the claim that "survival rates" (ie, five-year survival rates, the usual meaning of that term) are higher for prostate cancer in the US is not "just plain false", as Paul stated. It is in fact true. Misleading, as I tried to explain, but true.

Well then, why not look just at mortality rates? Because mortality rates vary with incidence.  If mortality rates for certain cancers are lower in America, that may be because the incidence of those cancers (driven by factors other than efficacy of treatment) is lower. Changes in mortality rates over time tell you something about a country's whole cancer-control regime. By themselves, cross-country comparisons of mortality rates say little about the quality of  treatment. Why not look at mortality rates adjusted for incidence? One problem here is that we only have figures for recorded incidence, not actual incidence. The US is good at detecting prostate cancer early.  It records vastly  higher incidence of the disease than Britain. Since the disease has about the same mortality in both countries, you might conclude that US treatment was vastly superior. That would be wrong, because America's incidence figures are inflated (or you could say that  Britain's are suppressed) by differences in screening.

Careful assessments of the efficacy of cancer treatment need to take five-year survival rates, mortality, recorded incidence and a host of other complications and confounding influences into account. Reliance on the five-year survival rate alone is rightly criticised--and the case of prostate cancer shows why--but it remains the standard measure. And on that measure, overall US cancer treatment scores better than Britain's.

Enough metric crunching. I said that the politically salient question is this: if you have cancer, would you rather be an American with health insurance or a Brit without private insurance, forced to rely on the NHS? I said the answer was obvious. If anybody with professional experience of the two systems thinks I'm mistaken about that, I'd love to hear from you.