Where are the readers in this discussion? Yes, scientists should do high-quality work and journalists should report it responsibly, but readers should be discerning and thoughtful information consumers. They can’t expect science writing to provide simple answers to complex questions; in fact, they should be skeptical of any piece that claims to do so. As University of Virginia psychology professor Brian Nosek told me, "Science reporting is not purveying the facts, it's purveying the discovery process, the adventure into the unknown."
Getting the most out of science writing takes work, but it’s vital, and similar to the attention we devote to consuming other products: We check the labels on food packages at the supermarket. We pore over online reviews before making even minor purchases. We should put the same care into the way we absorb scientific information, which has the power to shape the way we live.
To be a thoughtful reader, there are a few questions you should ask yourself whenever you read a popular science piece. I’ll use Hanna Rosin’s 2009 story in The Atlantic, about the evidence for and against the benefits of breastfeeding, as an example, because it addresses these questions especially well.
What’s the Big Picture?
A lot of science writing focuses on single studies. But each study is only a piece of the puzzle; look for writing that provides context. Before delving into the research on breastfeeding in her piece, Rosin provides historical background that helps explain why people have such strong feelings about breastfeeding and formula. She describes, for example, how some of the concern about formula stemmed from an international scandal in the 1970s, in which babies who were formula-fed in South America and Africa were more likely to die than those who were breastfed. It turned out that this was because mothers were using contaminated water or rationing formula because it was so expensive. Still, she writes, “the whole episode turned breast-feeding advocates and formula makers into Crips and Bloods, and introduced the take-no-prisoners turf war between them that continues to this day.”
Her review of the breastfeeding research is similarly nuanced. The article’s title, “The Case Against Breastfeeding,” suggests a diatribe, but the piece itself provides an even-handed review of the literature, describing studies that have found evidence for the benefits of breastfeeding and others that have not. In this way, she builds toward her overall conclusion: that claims about the benefits of breastfeeding have been overstated.
Even a good study has limitations and weaknesses. Usually the researchers are direct about these in academic papers, and do a decent job of explaining how they might detract from the study’s conclusions. Look for science writers to be similarly frank. Examples of weaknesses include: samples that are either very small (which makes it difficult to find a statistically significant difference) or very large (which means tiny, basically meaningless effects might still be statistically significant), or are somehow unrepresentative of the population they’re trying to understand (studying animals to learn about human diseases, for example).