The Ethics of Sarcastic Science
Every year the British Medical Journal publishes an issue of joke science. But years later, those papers are cited as real.
Every holiday season, the British Medical Journal puts out a special Christmas issue. It’s full of papers, as usual, but they’re all a little bit different. They’re jokes. Not fake—the data presented in these BMJ articles aren’t made up—but the premises of the papers are all a bit off-kilter. This year, for example, they showed that men die earlier than women because they’re stupid.
The BMJ has been loosening its ties every Christmas now for 30 years. In that time it has amassed a fair amount of odd little bits of science. But a recent paper on the subject of joke papers, by Lawrence Souder and his co-author Maryam Ronagh, questions whether these wacky studies are all in good fun, or whether there’s a darker side here. Ultimately, they argue that once the laughs have worn off, spoof papers can actually do damage to science.
Souder’s paper focuses on one case in particular. In 2001, Leonardo Leibovici published a paper titled “Effects of remote, retroactive intercessory prayer on outcomes in patients with bloodstream infection: Randomised controlled trial.” The study purported to show “whether remote, retroactive intercessory prayer, said for a group of patients with a bloodstream infection, has an effect on outcomes.” The study was farcical—the prayers they said for these patients were delivered between four and 10 years after their hospitalization. In some cases these prayers were said for them after they had already died. The reasoning for this, Leibovici explained, was that “we cannot assume a priori that time is linear, as we perceive it, or that God is limited by a linear time, as we are.”
Leibovici’s paper was one of many of BMJ’s Christmas spoofs, appearing in the journal alongside other joke articles. But eight years later the paper was cited, unironically, in a review paper from a well-respected organization.
And here we have the first issue with spoof articles. Once they’re published, they’re filed into the archives along with everything else, and they’re called up in searches as if they’re regular studies. As more researchers move away from reading journals in their paper form—in which an editorial or an opening letter from the editor might remind a reader of the nature of the pieces—the context in which they initially live is stripped away. This isn’t a problem just in science. On Facebook, there is a new “Satire” tag affixed to articles from The Onion. But there is no “Holiday Joke” tag available in Google Scholar or PubMed. People aren’t reading things in their original homes anymore, whether those things are journal articles or lists of snakes in top hats. And plucked away from a homepage and placed into a stream, whether that’s a Facebook timeline or a PubMed search result, means stripping away context. For cat videos it doesn’t generally matter. For joke science it does.
And Leibovici’s piece isn’t the only one to be cited later on without a hint of irony. I searched for a few past Christmas issue studies to see where they’ve showed up since. One joke study from 2007 on the energy expenditure of adolescents playing video games has been cited about 400 times since then, according to a Google Scholar estimate. In 2010 a paper called “Effect on gastric function and symptoms of drinking wine, black tea, or schnapps with a Swiss cheese fondue: randomised controlled crossover trial” that examined just how good or bad it is to drink alcohol with fondue was cited later in studies about wine and heart health, children with ADHD, and gastrointestinal gallbladder emptying. A study called “sex, aggression, and humor: responses to unicycling” was cited in 2012 as evidence for “the evolution of humor from male aggression” and appears in a book called The Male Brain.
In fact, that unicycle study wasn’t just cited by other scientists, it was picked up by the BBC for a story with the headline “Humour ‘comes from testosterone.’” And here is a good example of another issue with these parody pieces. They often fail a basic premise of comedy: Punch up, not down. Using big science words to make fun of women and those who believe in prayer, even if it’s a parody, isn’t funny, it’s sexist and elitist. And formalizing those jokes in a scientific journal such that only those intimately familiar with the schedule that journal keeps (Christmas is for comedy papers, didn’t you know?) is a nice, formalized way to exclude people.
Souder points to a quote from Wayne Booth, author of A Rhetoric of Irony. Booth writes,
There is reason to believe that most of us think we are less vulnerable to mistakes with irony than we are. If we have enjoyed many ironies and observed less experienced readers making fools of themselves, we can hardly resist flattering ourselves for making our way pretty well.
And that is, in many ways, what’s going on here. Spoof journal articles are a way of solidifying who is “in” and who is “out.” Who gets the joke, and who doesn’t. At a time when scientists crave acceptance, when they bemoan the public’s lack of interest or support, perhaps it’s worth examining these jokes and wondering whether they might be part of the problem.
Not everyone agrees. At Discover, the blogger Neuroskeptic points out that, like all BMJ holiday parodies, Leibovici’s paper didn’t lie, and it didn’t make anything up. So those who cite the paper aren’t doing so in error, they’re not being fooled or lied to. “If someone believes in prayer changing the past, then they believe something absurd. Citing this paper in support of their belief doesn’t make them more absurd, however,” Neuroskeptic argues. I’d argue that that’s missing the forest through the trees. It’s not a matter of truth, it’s a matter of trust.
Souder says that as an editor himself, he’s less interested in a scientist having a good chuckle at the expense of outsiders, and more interested in maintaining a reader’s loyalty and confidence. “I’m always concerned about the reader,” he says. “I’m an editor myself so I feel like I’m the reader’s advocate.” He sums up his worry in the paper this way: “Readers of scientific research will not expect a published paper to express certainty about the truth of its content, but they should expect certainty about the truthful intent of its authors.” Science isn’t truth, Souder says—it can’t be—and expecting the whole truth from a scientific paper is to forget how science is done, in fits and starts, rewriting itself continuously. But readers should be able to expect that the authors are aiming for truth, pointing their cannon at it. And they should be able to expect that without constantly checking the date on a paper.
This isn’t to say that joke science is off-limits. Souder is not the science joke police, here to take away everybody’s hilarious jargon-based punchlines. But he does think that there should be some method of ensuring that it’s clear to everybody, not just those “in the know,” when an article is a parody and when it is not. Journals could go the Facebook route, labeling each Christmas issue with a little “Parody” tag. It’s not unlike handling a retraction, Souder said, by putting a disclaimer at the top explaining the caveats and concerns surrounding the paper. But Souder admits that kind of ruins the joke. “You’re undercutting the irony at that point,” he says. “Saying, ‘Oh by the way, this is ironic, okay?’ I understand the problem with that.”
What Souder and Ronagh propose in their paper is something a little different: indexing joke papers differently than serious ones. “You could have them tagged in a way that wasn’t obvious to the reader but was obvious to the machine that indexes these things, so that they don’t come up when you do a search.”
There is no blog chronicling those who accidentally take joke science seriously, as there is for The Onion. But it's also less of a problem when your uncle thinks a woman really ate an entire live tuna on a city bus.