The Myth of Self-Correcting Science

More

Recent academic scandals highlight a history of data falsification and questionable research in social psychology, and serve as calls to action. 

RTR36NSS615.png
Carlos Jasso/Reuters

Over the last two years, the field of psychology has endured a wave of scandal bookended by fraud cases involving Harvard primatologist Marc Hauser and Dutch social psychologist Diederik Stapel. Even researchers desensitized by scandal-fatigue did a double take when the final report on Stapel's case came out last month. The extent of his creative misinterpretation of the facts make the Hauser case look like child's play. Stapel not only manipulated and fabricated data, he invented entire schools where said data was allegedly collected.

As if the fraud files weren't enough, then come the mea culpas -- salt in the wounds for students and colleagues still recovering from shattered reputations and a shaken faith in science. The two men released two very different statements telling very similar stories of reckless, ruthless ambition and playing the odds against getting caught. Stapel's "narcissistic wail" was so emotional and contrite as to seem a bit unhinged, while Hauser's read as a cold, calculating non-admission of guilt.

Hauser deftly concedes chagrin for errors made within his lab "whether responsible for them or not," implying that the same students bullied into committing academic fraud were somehow responsible for the car veering off the cliff. Stapel faults a noxious combination of publication pressures, addictive tendencies, and assorted personality issues for his downfall. And while publication pressure was among those issues, he caps off his mea culpa with a plug for his new book -- Derailment, a collection of his therapeutic diaries.

The Slippery Slope

It's easy to revel in the high drama surrounding the downfall of a Hauser or Stapel, but what about the journals that published these scholars? Stapel was a widely cited and highly revered figure. His fraud went undetected for decades in spite of eerily perfect data sets and improbable statistical values. According to Tilburg University's final report, Flawed Science, "There was a general neglect of fundamental scientific standards and methodological requirements from top to bottom."

Scientists fought back, noting that it is rare for reviewers in any field to detect fraud and demanding an apology for the 'slanderous conclusions' drawn in the report. Social psychologist Kate Ratliff, teaching at Tilburg when the scandal broke noted, " It's a small community and people considered Diederik a friend and mentor...No one understands why these young researchers didn't realize that it was weird that Diederik was giving them datasets. But you learn from watching others. And if there are no others, how would you know what's weird or not? I think that people started out being really sympathetic toward them and have gotten more and more punitive as time passes and hindsight bias kicks in. I think that's really, really unfair."

They managed to find statistically significant evidence for the absurd hypothesis that listening to a Beatles song could make you 1.5 years younger

Almost more alarming than the few individuals committing academic fraud are the high percentage of researchers who admitted to more common questionable research practices, like post-hoc theorizing and data-fishing (sometimes referred to as p-hacking), in a recent study led by Leslie John.

For the uninitiated: post-hoc theorizing involves creating or revising a hypothesis after you've collected the data; data-fishing entails running a study, continually checking the data after each participant, and stopping as soon as you see a significant result. These practices are eschewed by some, but plenty of others embrace them. Joseph Simmons and colleagues ran a simulation showing how unacceptably easy it was to attain statistical significance using these 'degrees of researcher freedom.' By employing four of these questionable practices at once, they managed to find statistically significant evidence for the absurd hypothesis that listening to a Beatles song could make you 1.5 years younger.

"Clearer identification of the problems associated with some research practices is incredibly helpful," writes Linda Skitka, who sits on numerous journal editorial boards. "Because I'm guessing at least some scholars who engaged in questionable practices did not recognize the full implications of doing so. Given the intense attention these issues are now getting in the field, they certainly know better now."

So are the social sciences more prone to misconduct and fraud than biomedicine and other fields? A recent study titled " Scientific Misconduct and the Myth of Self-Correction in Science" found no such evidence. Even Stapel's wildly narcissistic mea culpa can't make you forget Yoshitaka Fujii, the Japanese anesthesiologist with a record-breaking 172 retractions.

Biomedicine shares some of the more nebulous concerns regarding data transparency, collection and dissemination as well. Citing the current drama surrounding Tamiflu , Nick Genes notes, "This is a hot topic [in medicine] right now ... There's a movement to bypass what's published and dive into the original data that's kept by drug companies and/or given to regulatory agencies like the FDA." Much as with the social sciences, the raw data from clinical trials is not made publicly available, and many fear that the temptation to tell a self-serving story with the data in journal articles (for individuals or pharmaceutical companies) will be too great.

F1.large.jpg


The Old Guard and the New

Though a wave of ignominy is cresting at the moment, these problems are not new. Back in the "golden era" of the NIH during the fifties and sixties, David Guston writes, if fraud occurred, the director would make a few phone calls, look into the alleged misconduct, and the offending scientist would be "quickly and quietly removed from the map of science." At that time, "the social contract for science was highly informal and contained entirely within the community."

Jump to comments
Presented by

Sarah Estes is a fellow with To Think, to Write, to Publish, which is funded by the National Science Foundation, and a regular contributor to Scientific American and Psychology Today.

Get Today's Top Stories in Your Inbox (preview)

Is Technology Making Us Better Storytellers?

How have stories changed in the age of social media? The minds behind House of Cards, This American Life, and The Moth discuss.


Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus

Video

Is Technology Making Us Better Storytellers?

The minds behind House of Cards and The Moth weigh in.

Video

A Short Film That Skewers Hollywood

A studio executive concocts an animated blockbuster. Who cares about the story?

Video

In Online Dating, Everyone's a Little Bit Racist

The co-founder of OKCupid shares findings from his analysis of millions of users' data.

Video

What Is a Sandwich?

We're overthinking sandwiches, so you don't have to.

Video

Let's Talk About Not Smoking

Why does smoking maintain its allure? James Hamblin seeks the wisdom of a cool person.

Writers

Up
Down

More in Health

Just In