Border and Keller have heard that argument before. So, in their study, they measured depression in many ways—diagnosis, severity, symptom count, episode count—and they accounted for environmental factors such as childhood trauma, adulthood trauma, and socioeconomic adversity. It didn’t matter. No candidate gene influenced depression risk in any environment.
But Suzanne Vrshek-Schallhorn of the University of North Carolina at Greensboro says that Border’s team didn’t assess life experiences with enough precision. “I cannot emphasize enough how insufficient the measures of the environment used in this investigation were,” she says. “Even for measures that fall below gold-standard stress-assessment approaches, they represent a new low.” By using overly simple yes-or-no questionnaires rather than more thorough interviews, the team may have completely obscured any relationships between genes and environments, Vrshek-Schallhorn claims. “We should not get starry-eyed about large sample sizes, when measure validity is compromised to achieve them. We need to emphasize both quality and quantity.”
But Border argues that even if there had been “catastrophic measurement error,” his results would stand. In simulations, even when he replaced half the depression diagnoses and half the records of personal trauma with coin flips, the study would have been large enough to detect the kinds of effects seen in the early candidate-gene papers.
Similar debates have played out in other fields. When one group of psychologists started trying to reproduce classic results in much larger studies, their peers argued that any failures might simply be due to differences between the new groups of volunteers and the originals. This excuse has eroded with time, but to Border, it feels familiar. “There’s an unwillingness to part with a previous hypothesis,” he says. “It’s hard to wrap your head around the fact that maybe you were on a wild goose chase for years.”
Keller worries that these problems will be used as ammunition to distrust science as a whole. “People ask, Well, if scientists are publishing crap, why should we believe global warming and evolution?” he says. “But there’s a real difference: Some people were skeptical about candidate genes even back in the 1990s. There was never unanimity or consensus in the way there is for human-made global warming and the theory of evolution.”
Nor, he says, should his work be taken to mean that genes don’t affect depression. They do, and with newer, bigger studies, researchers are finally working out which ones do. If anything, the sordid history of the candidate-gene approach propelled the development of better methods. “I feel like the field of psychiatric genetics felt really burned coming out of the candidate-gene era, and took strides to make sure it won’t happen again.” That includes sharing data openly, and setting standards for how large and powerful studies need to be.