Make Science More Reliable, Win Cash Prizes

Honoring young researchers who champion rigorous, transparent research is a small step towards changing the culture of science.

Some scientific prizes reward people for their skills as mentors, or for the quality of their scientific discoveries. But the new Leamer-Rosenthal Prizes for Open Social Science, awarded today for the first time, honor a different kind of achievement: keeping science itself honest.

Created by the Berkeley Initiative for Transparency in the Social Sciences (BITSS), the $10,000 prizes recognize social scientists who have contributed to the process of science, either by developing tools to allow others to carry out rigorous research, or by exemplifying such research themselves.

They come at a time when many psychologists, economists, and other social scientists are increasingly concerned about the reliability of results in their field, following widespread failures to replicate published studies, evidence of sloppy but commonly used research practices, and cases of outright fraud. Many researchers are working on innovative solutions to the so-called replicability crisis, including pre-registering experiments before they are done, setting standards for openly sharing methods and data, and even establishing a futures market for science.

The problem is that there are no rewards for taking part in these efforts. Instead, modern science, with its emphasis on publishing in top-tier journals, rewards researchers for making discoveries that are new and surprising—but not necessarily true. “These incentive structures will be slow to change,” says Temina Madon, the director of the BITSS. “We wanted to do something more immediate, to circumvent the very slow process of reform in academia.”

Ten winners were selected from 51 nominees. “They’re a really exciting group of creative activist-scientists, whom I think others can aspire to,” says Madon. “From the get-go, the vision was to focus on younger researchers. They take the greatest risks in engaging in transparent processes, because it makes their work more vulnerable to critique. We wanted to reward them.”

David Broockman, Joshua Kalla, and Peter Aronow, who jointly won one of the Emerging Researcher prizes, certainly took risks. In analyzing a highly publicized study on swaying attitudes to gay marriage, the trio showed that the political scientist Michael LaCour had fabricated the data behind the study—which has since been retracted. “As a young academic with a freshly minted Ph.D., awards like this definitely send the signal to people like me that standing up for transparent practices will be rewarded,” says Kroopman.

Other winners include: Etienne LeBel, whose CurateScience platform makes it easier for researchers to check each others’ results; Dora Matzke, who created a consulting service called Stats Store that provides researchers with statistical and methodological advice; and J. Scott Long, who has written several textbooks on reproducibility.

Eva Vivalt from Stanford University, another winner, founded a nonprofit organization called AidGrade to determines what works in international development. By collating the results from disparate evaluations, and analyzing them together, AidGrade provides open assessments about which programs and organizations are actually working.

“I think it is vitally important to have rewards for this kind of work, because researchers often do not have the incentive to do research relating to transparency or to be transparent about what they did,” says Vivalt, who plans to donate her prize to AidGrade.

The mere fact that such prizes exist makes an important statement, says Brian Nosek from the Center for Open Science. “They signal that the research community cares about transparency and reproducibility. They’re the first step in a cultural shift of incentives for open practices, and they’re first because they are easy to establish, and provide high-profile bang-for-the-buck in signaling norms and values.”