After the Latest MH370 Report, How to Think About Speculation

By James Fallows

Once again I am spending most of today in transit, and the doors of a (commercial) flight are about to close. So this is a placeholder note about today's announcement from the Malaysian prime minister about the fate of Malaysia Air flight 370.

The most urgent concerns about this flight are of course those of the families affected. Next come questions about this airline, this model of aircraft, the air-traffic-control and air-safety operations in this part of the world, and any other potential source of longer-term systematic problems. 

And then there is the question about how our news media deal with the unknowable. The human fascination with the fate of this flight is understandable and natural. So far it has been a mystery with no obvious precedent in the world of modern air travel.

Structurally, these past few weeks have also reinforced an obvious point about 24-hour cable news. Its basic premise is that what is happening right now is more important and compelling than whatever may have been regularly programmed on some other channel—or in the other aspects of your life. Thus it is natural that CNN, in particular, has gone wall-to-wall with coverage even on days with no development resembling actual news. This is the war-style coverage that gave CNN its start, applied to a different situation.

In terms of the content of news-while-waiting-for-news, I've come to value the analysts, panelists, "experts," and others who display two traits:

* They have emphasized the unknowability of the entire situation, the contradictory nature of much "evidence," and the tentativeness of assumptions about what could and could not have happened with MH370.

* At the same time, they have helped the public separate the possible-but-unconfirmed from the FantasyLand-wild improbabilities. The clearest indication of this last category is the "radar shadow" hypothesis, which I'll link to later. Or a prominent official's straight-faced assertion that the plane might be headed to Israel on an attack mission.

Two of the panelists who consistently met this test were Les Abend, a former commercial pilot, and Miles O'Brien, a TV veteran and (Cirrus) pilot. There may have been others, but they were the ones I saw most consistently talking sense.

Now, on the general phenomenon of speculating about the improbable, I give you this note from several days ago from reader T.J. Radcliffe, a scientist. He writes:

Your observation "modern airlines are so extraordinarily safe that when something goes wrong, the full story is usually by definition unusual" is precisely why speculation ahead of the data is so utterly irresponsible in this case.

The famous dictum "when you hear hoof-beats, think horses no zebras" only works because horses are a single, fairly common source of hoof-beats in everyday life.

For any given effect, good Bayesians are bound to focus their speculations on things that are about as probable as the most common cause. But in the case of modern air disasters, due to the amazing gains in safety in recent decades, the most common cause is always wildly improbable, which means there are an almost infinite number of alternatives of equally low probability. So speculation is pointless, and the people who do it are being flagrantly non-Bayesian, which never ends well.

Speculation, like imagination generally, is not a particularly effective tool for deciding what is true. We can and do imagine impossible things (perpetual motion machines, political violence that actually achieves its purported end) and we fail to imagine things that actually exist (evolution by variation and natural selection, creative non-violence as a viable strategy for political change). The human imagination is many things, but as a tool for knowing it's about as good as a hammer for tightening bolts.

For low-probability events when very little data is available, our ugly tendency to fall back on our imagination comes to the fore on all sides, as it has in the past two weeks with MH370. While my heart goes out to the loved ones of the passengers, that some of them cannot imagine how difficult it is to find wreckage in mid-ocean does not justify their harassment of public officials, who are no-doubt struggling with feelings of inadequacy due to their own lack of understanding of the laws of probability.

There is a Kuhnian revolution going on right now in our understanding of the world in terms of probability distributions rather than mechanical, binary, cause and effect. It will take decades or centuries for this to percolate through society, but the foolishness of non-Bayesian speculators in the case of MH370 is one more example of how the old ways fail all of us in times of crisis and heartbreak.

He has a related post on his own site, here

This article available online at:

http://www.theatlantic.com/technology/archive/2014/03/after-the-latest-mh370-report-how-to-think-about-speculation/284610/