The Myth of Self-Correcting Science

More

Recent academic scandals highlight a history of data falsification and questionable research in social psychology, and serve as calls to action. 

RTR36NSS615.png
Carlos Jasso/Reuters

Over the last two years, the field of psychology has endured a wave of scandal bookended by fraud cases involving Harvard primatologist Marc Hauser and Dutch social psychologist Diederik Stapel. Even researchers desensitized by scandal-fatigue did a double take when the final report on Stapel's case came out last month. The extent of his creative misinterpretation of the facts make the Hauser case look like child's play. Stapel not only manipulated and fabricated data, he invented entire schools where said data was allegedly collected.

As if the fraud files weren't enough, then come the mea culpas -- salt in the wounds for students and colleagues still recovering from shattered reputations and a shaken faith in science. The two men released two very different statements telling very similar stories of reckless, ruthless ambition and playing the odds against getting caught. Stapel's "narcissistic wail" was so emotional and contrite as to seem a bit unhinged, while Hauser's read as a cold, calculating non-admission of guilt.

Hauser deftly concedes chagrin for errors made within his lab "whether responsible for them or not," implying that the same students bullied into committing academic fraud were somehow responsible for the car veering off the cliff. Stapel faults a noxious combination of publication pressures, addictive tendencies, and assorted personality issues for his downfall. And while publication pressure was among those issues, he caps off his mea culpa with a plug for his new book -- Derailment, a collection of his therapeutic diaries.

The Slippery Slope

It's easy to revel in the high drama surrounding the downfall of a Hauser or Stapel, but what about the journals that published these scholars? Stapel was a widely cited and highly revered figure. His fraud went undetected for decades in spite of eerily perfect data sets and improbable statistical values. According to Tilburg University's final report, Flawed Science, "There was a general neglect of fundamental scientific standards and methodological requirements from top to bottom."

Scientists fought back, noting that it is rare for reviewers in any field to detect fraud and demanding an apology for the 'slanderous conclusions' drawn in the report. Social psychologist Kate Ratliff, teaching at Tilburg when the scandal broke noted, " It's a small community and people considered Diederik a friend and mentor...No one understands why these young researchers didn't realize that it was weird that Diederik was giving them datasets. But you learn from watching others. And if there are no others, how would you know what's weird or not? I think that people started out being really sympathetic toward them and have gotten more and more punitive as time passes and hindsight bias kicks in. I think that's really, really unfair."

They managed to find statistically significant evidence for the absurd hypothesis that listening to a Beatles song could make you 1.5 years younger

Almost more alarming than the few individuals committing academic fraud are the high percentage of researchers who admitted to more common questionable research practices, like post-hoc theorizing and data-fishing (sometimes referred to as p-hacking), in a recent study led by Leslie John.

For the uninitiated: post-hoc theorizing involves creating or revising a hypothesis after you've collected the data; data-fishing entails running a study, continually checking the data after each participant, and stopping as soon as you see a significant result. These practices are eschewed by some, but plenty of others embrace them. Joseph Simmons and colleagues ran a simulation showing how unacceptably easy it was to attain statistical significance using these 'degrees of researcher freedom.' By employing four of these questionable practices at once, they managed to find statistically significant evidence for the absurd hypothesis that listening to a Beatles song could make you 1.5 years younger.

"Clearer identification of the problems associated with some research practices is incredibly helpful," writes Linda Skitka, who sits on numerous journal editorial boards. "Because I'm guessing at least some scholars who engaged in questionable practices did not recognize the full implications of doing so. Given the intense attention these issues are now getting in the field, they certainly know better now."

So are the social sciences more prone to misconduct and fraud than biomedicine and other fields? A recent study titled " Scientific Misconduct and the Myth of Self-Correction in Science" found no such evidence. Even Stapel's wildly narcissistic mea culpa can't make you forget Yoshitaka Fujii, the Japanese anesthesiologist with a record-breaking 172 retractions.

Biomedicine shares some of the more nebulous concerns regarding data transparency, collection and dissemination as well. Citing the current drama surrounding Tamiflu , Nick Genes notes, "This is a hot topic [in medicine] right now ... There's a movement to bypass what's published and dive into the original data that's kept by drug companies and/or given to regulatory agencies like the FDA." Much as with the social sciences, the raw data from clinical trials is not made publicly available, and many fear that the temptation to tell a self-serving story with the data in journal articles (for individuals or pharmaceutical companies) will be too great.

F1.large.jpg


The Old Guard and the New

Though a wave of ignominy is cresting at the moment, these problems are not new. Back in the "golden era" of the NIH during the fifties and sixties, David Guston writes, if fraud occurred, the director would make a few phone calls, look into the alleged misconduct, and the offending scientist would be "quickly and quietly removed from the map of science." At that time, "the social contract for science was highly informal and contained entirely within the community."

Jump to comments
Presented by

Sarah Estes is a fellow with To Think, to Write, to Publish, which is funded by the National Science Foundation, and a regular contributor to Scientific American and Psychology Today.

Get Today's Top Stories in Your Inbox (preview)

Why Did I Study Physics?

In this hand-drawn animation, a college graduate explains why she chose her major—and what it taught her about herself.


Elsewhere on the web

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register. blog comments powered by Disqus

Video

Why Did I Study Physics?

Using hand-drawn cartoons to explain an academic passion

Video

What If Emoji Lived Among Us?

A whimsical ad imagines what life would be like if emoji were real.

Video

Living Alone on a Sailboat

"If you think I'm a dirtbag, then you don't understand the lifestyle."

Video

How Is Social Media Changing Journalism?

How new platforms are transforming radio, TV, print, and digital

Video

The Place Where Silent Movies Sing

How an antique, wind-powered pipe organ brings films to life

Feature

The Future of Iced Coffee

Are artisan businesses like Blue Bottle doomed to fail when they go mainstream?

Writers

Up
Down

More in Health

Just In