Why a Study on Opioids Ignited a Twitter Firestorm

A paper on overdose-reversal drugs reached a conclusion no one liked. The pushback raised questions about sexism and scientific methods.

Police Detective Lt. Patrick Glynn holds a nasal injection containing naloxone in 2014.
Police Detective Lt. Patrick Glynn holds a nasal injection containing naloxone in 2014. (Gretchen Ertl / Reuters)

With the opioid epidemic claiming more than 100 lives a day in the United States, every state now has some sort of law expanding access to naloxone, also known as Narcan. Naloxone is an opioid antagonist that makes someone who has overdosed start breathing again. Sometimes, its powers are said to bring an overdose victim “back to life.” That led two economists to wonder, does the prospect of not dying from opioids make people more likely to use opioids? And are they more likely to, ultimately, die as a result?

The two researchers—Jennifer Doleac, of the University of Virginia, and Anita Mukherjee, of the University of Wisconsin—looked at the time period before and after different naloxone-access laws were put into place, such as providing legal immunity to people who prescribed or administered the drug and allowing anyone to buy naloxone in a pharmacy without a prescription.

After naloxone-access laws take effect, they found, arrests related to the possession and sale of opioids went up, as did opioid-related ER visits. Meanwhile—and most worryingly—there was no overall impact on the death rate. In fact, in the Midwest, the implementation of naloxone laws led to a 14 percent increase in opioid-related mortality, they found.

To Doleac, it’s an example of moral hazard. In other words, “anytime you make something less dangerous, people are going to do more of it,” she told me. In one study, for example, giving teens free condoms actually led to an increase in teen pregnancy. In this case, the study purports that heroin users figured they stood a good chance of being revived if they overdosed, so they kept on using.

Doleac announced her findings with a Twitter thread:

She was not quite “ratioed,” to use the digerati term for a tweet that garners more responses than approvals. But she nevertheless prompted a vigorous online debate that raged for nearly a week. In the white-paper reading realm, that’s practically World War III.

Public-health people were, perhaps understandably, alarmed by Doleac and Mukherjee’s findings, suggesting they might lead cities and states to pull back from providing naloxone freely. Naloxone access is considered a pillar of “harm reduction,” or the idea that if people can’t immediately be cured of addiction, we should at least make it less dangerous for them to keep using.

As with almost every study, there are other studies that came to different conclusions. One 2012 study in Massachusetts found that opioid-overdose death rates were reduced in communities where people were given nasal-naloxone rescue kits, and another paper last year came to a similar conclusion. One study found naloxone did not lead to increased heroin use.

Leana Wen, Baltimore’s health commissioner and an advocate for the expansion of access to naloxone, pointed out that just because the laws around naloxone changed doesn’t mean people were able to instantly obtain it more easily. “[Doleac and Mukherjee’s] study assumes that passage of these laws lead immediately to everyone having easy access to naloxone when they need it, when this is not the case,” she says. “The Massachusetts study shows that actual naloxone distribution shows a decrease in mortality. This is consistent with what we see in Baltimore, where over 1,600 lives have been saved as a result of our naloxone distribution program.”

Wen also said that even though there was no reduction in opioid-related mortality, the death rate might have been even higher without naloxone.

Several critics of the study claimed it is yet another example of the classic problem of confusing correlation with causation. “The first problem is that they seem to be making the big mistake of assuming that correlation equals causation,” says Jermaine Jones, a Columbia University neurobiologist whose study found naloxone didn’t increase heroin use. “This misinterpretation of data is one of the first things we are taught in psychology.” Jones went on to tell me about the classic example of ice-cream sales and murder rates. Of course, the sugar high doesn’t spark killing rampages; the hidden variable there is summertime.

But Doleac and other economists say this is one of the differences between public-health research and economics. Economists often make “causal inferences”: They study natural experiments like law changes, use statistical tools to rule out other explanations, and draw conclusions about cause and effect. (Doleac, in turn, criticized the methods used in the studies that came to different conclusions than her own.)

“Public-health people believe things that are not randomized are correlative,” says Craig Garthwaite, a health economist at Northwestern University. “But [economists] have developed tools to make causal claims from nonrandomized data.”

This difference of opinion about causality led to exchanges like this one, between one of Doleac’s critics and another female economist, Analisa Packham:

The other thing critics brought up is that Doleac and Mukherjee’s article is not yet peer reviewed—they plan to submit it to journals soon. But this, too, is an important difference between economics and some other types of health research. Economists tend to put out working papers and circulate them among colleagues long before they submit to journals. For example, Doleac and Mukherjee’s paper thanks several economic conferences and their participants in its acknowledgments. These colleagues—sometimes in pressure-cooker-esque seminars—ask questions and make suggestions, after which the paper is revised, then submitted.

Though this is not quite the same as a journal’s peer-review process, it’s not quite a personal Word document uploaded to the internet, either. Because of that, economics working papers are regularly discussed by journalists, academics, and even policymakers before they’re formally published in a journal.

What’s more, the Twitter conversation at times turned unusually venomous and condescending, especially for an economics paper.

In some cases, it seemed more suited to, say, debates about putting peas in guacamole or the potential outcomes of single-payer health care. The tone “felt sort of accusatory and ‘let me explain to you simple authors about how this works,’” Garthwaite says.

One person, for example, called Doleac “a ghoul:”

Another implied that Doleac hates poor people. (It’s worth noting, of course, that opioid addicts aren’t always poor.)

It led some to wonder how much of the pushback was about peoples’ interest in the opioid epidemic and how much was simply what happens when female researchers’ findings don’t quite toe the party line.

All of this isn’t to say that Doleac and Mukherjee’s findings are the last word on the matter, or that they shouldn’t be scrutinized and (respectfully) questioned. They might still hit snags in peer review, or another study might come along and refute their findings. Such is the nature of science.

But it does suggest two things: First, more research is (almost) always good. Doleac told me she tried to replicate an earlier study on naloxone and mortality, and she failed. Maybe someone else might try to replicate her work and fail, too.

Second, when studies come along that find something we don’t like, the instinct can be to flinch and look away—or worse, to insult the study author. But it’s important not to dismiss findings like these, because if they hold up, they might tell us how to better shape public policy.

The other famous example of moral hazard involves car-safety regulations: Safer cars make people drive more recklessly, according to an infamous 1970s study. But, Garthwaite told me, “no one who’s serious would say that just because moral hazard exists means we shouldn’t put seat belts in cars.” In this case, it might mean that cities should dispense naloxone, but that they should also invest more in treatment—something Doleac and Mukherjee themselves call for in this study. Naloxone, Garthwaite says, might be a good intervention, but it might not be a good intervention on its own.

Doleac says this was the last round of comments she was hoping to get before she submits to peer review. “And we have gotten it,” she added, in the form of the hundreds of tweets and emails.

Doleac is no stranger to controversy. Previously, she studied racism and gun violence. “The response to this has been so much more negative and hostile than anything else I’ve worked on,” she says. She says one doctor emailed her to explain how to write an academic paper. “I was like, ‘I’ve actually done this before!’”

She also says she hasn’t heard anything that would change the overall conclusions of the study. “We might clarify some things in the text,” but those changes, she acknowledges, “probably won’t make anyone happy.”

Something tells me we don’t need peer review to know she’s probably right about that.