'One of the Greatest Discoveries in the History of Science' Hasn't Been Peer-Reviewed—Does It Matter?

By Megan Garber

Big scientific discoveries—the kind that shift our view of the world and our place within it—don’t come along very often.

This week, though, one did.

New data seem to offer, for the first time, direct evidence of the entities Einstein predicted in his general theory of relativity: gravitational waves. Which is a finding that, if it holds up, sheds new light on nothing less than the origins of the universe. The discovery is, according to one expert, “an amazing achievement.” It is also, according to another, “one of the greatest discoveries in the history of science”—“a sensational breakthrough involving not only our cosmic origins, but also the nature of space.”

So, basically: This is big, you guys! Einstein big! Nature-of-space big! Big Bang-big!

There’s just one small thing, though. The findings shared this week also share a significant caveat: They haven’t yet been peer-reviewed. They are discoveries that are, as far as scientific institutionalism is concerned, provisional. They’re stuck in a kind of epistemological limbo—as information that has not yet been converted into fact, and data that have not yet been codified into knowledge. Official status: truthy.

***

Scientists are, like the rest of us, impatient. They are, much more often than the rest of us, justified in this. Imagine dedicating your career to learning something new about the mechanics of the world—the gravitational forces exerted on a cell membrane, the flappings of a bee’s wings, the earliest churnings of the cosmos—and then imagine actually finding that thing. Now imagine that, instead of doing what every impulse would guide you to do (share that news with everyone you know/share that news with everyone you don't know/shout that news from the rooftops or at least your Facebook page) … you are made to wait. And wait. And wait. Until, many months later, your work has been deemed acceptable for proper publication.

For many scientists, this holding pattern of human enthusiasm is one of the most salient facts of peer review, the painstaking process by which the discoveries of individual scientists are weighed and tested by fellow experts in the field. It's a process that is purposely inefficient and pointedly complex, one meant to distinguish scientific discoveries from every other kind. It is also, in an age when anyone—scientists included—can be a publisher, increasingly controversial. A 2011 report commissioned by the British House of Commons found that, while peer review "has always been regarded as crucial to the reputation and reliability of scientific research” and “continues to play an important role in ensuring that the scientific record is sound,” many scientists also find its constraints to be detrimental to their work and their ability to share it. (Some also doubt its worth more generally: There is, the report noted, “little solid evidence” about peer review’s overall efficacy.)

Peer review, though as a concept it dates back to the Scientific Revolution, is not a fixed feature of the scientific method. The peer-reviewed scientific journal, the Harvard historian Melinda Baldwin points out, “isn't nearly as old as most observers think it is.” While “many histories of journal publishing claim that we've had both journals and peer review since the Scientific Revolution,” she told me, the highly specialized scientific journals we’re familiar with today “didn't become the dominant way of communicating scientific findings until the 19th century.” And “it wasn't until the 20th that journals had to be peer-reviewed to be considered scientifically respectable.” The prestigious journal Nature, she notes, made an occasional practice of publishing non-peer-reviewed research articles up through 1973.

Which is all to say: Just as scientific knowledge evolves, so do the tools that help us earn it. The mechanisms we rely on to encode the truths of the physical world evolve along with everything else. 

***

This week’s big Big Bang announcement is akin to the 2012 announcement of the discovery of the Higgs boson—which was both similarly epic (“God particle,” etc.), and similarly publicized before the scientific community had given it its institutional ratification. Before proceeding through peer review and journal publication (which would culminate in a Nobel Prize for Peter Higgs and his colleague, Francois Englert), the discovery was rushed to market. The market being, in this case, the media and the public at large.

It took, in the end, about two months—a relatively quick turnaround—for the Higgs's "potential Huge Discovery" to be upgraded to “actual Huge Discovery.” Which lead, eventually, to awkward announcements like this one: "CERN's Higgs boson discovery passes peer review, becomes actual science.” To the lay reader, a headline like that would seem confusingly redundant. 

So what we have in the Big Bang news, basically, is the same thing we had with the Higgs news: a disconnect between traditional systems of scientific codification and newer, nimbler ones—a tension between publicity strategies within scientific institutions and beyond them. It's not as stark a dichotomy as journal publication-vs.-web publication ... but it's close. There are, after all, competing pressures at play: on the one hand, the pressure to be first; on the other, the pressure to be right. And the latter pressure, ideally, trumps the former. Take the announcement of cold fusion (or, more accurately, "cold fusion"); in 1989, the chemists Stanley Pons and Martin Fleischman announced—to great fanfare, and via a widely publicized press conference—that they had achieved nuclear fusion at room temperature. In a jar of water. This was hailed as "the greatest discovery since fire" ... until none of the chemists' colleagues were able to replicate their work. Now the whole thing is treated as a cautionary tale.

Compounding these pressures in the case of the Big Bang news is the fact that physics in particular is a notoriously competitive field; its researchers are racing each other, always, to discovery. (This also makes it a notoriously secretive field: Bjorn Carey, Stanford's science information officer, told me that the Big Bang news was considered "top secret" until it was shared publicly.)

The speed imperative leads to tensions in many other fields beyond science (journalism certainly among them); we're all, in our own ways, figuring out how to navigate this platform for instant publication we now have at our disposal. The Big Bang news is simply emblematic of a larger trend. As the philosopher David Weinberger puts it: “Scientific knowledge is taking on properties of its new medium, becoming like the network in which it lives.”

So the web, with its anyone-can-publish capacities, may well be aiding a movement away from traditional—which is to say, unassisted—peer review. In addition to, and occasionally instead of, embracing the intentional inefficiencies of the peer-to-publish approach to acceptance, many scientists are posting their findings directly to sites like arXiv.org, the online repository of yet-to-be-published scholarly research. ArXiv is a popular site, and a regular in the RSS feeds of many science journalists; to publish a study on its platform is to invite media coverage for its findings, peer-reviewed or (as is usually the case) not.

There are also, with the help of university press offices and their increasingly sophisticated adoption of social media platforms, more deliberate invitations to public dissemination of work that has yet to be peer-reviewed. This week's Big Bang news, as befits its bigness, came to us via, among other avenues: press conference, web-blasted press release, web-based news article, and Stanford-produced YouTube video. This was scientific discovery at its most publicly tentacular.

Which may also be scientific discovery at its most productive. “I think people are recognizing that, in this age of very exchangeable information, making things as public as possible is a really good idea,” says Samuel Arbesman, a network scientist and the author of the book The Half-Life of Facts: Why Everything We Know Has an Expiration Date. “In science, it used to be that you wrote books,” he told me: You compiled your findings, you published them, and—BOOM—your results had a public life. And if you wanted to communicate those findings with the world more rapidly than a book would allow …  you were pretty much out of luck. “So the scientific paper,” Arbesman says, “was the response to trying to make sure you could actually publish bite-sized bits of knowledge and discovery more rapidly. And it was great technology.”

***

These transformations in media affordances are colliding with another pressure scientists face—for funding. Most researchers cannot be merely researchers; they must also be salespeople. With the products they're selling being themselves and their own work. As Declan Fahy, a science historian at American University, points out: “Scientific organizations are increasingly being forced to justify their existence and justify their funding.” And that, in turn, gives them an extra incentive to publicize their work outside of the traditional (and widely commented upon) constraints of journal publication. Scientific discovery is, increasingly, a public relations game.

It’s been that way, to be clear, since long before the Internet settled into widespread use. In 1985, the Royal Society in Britain published a document—The Public Understanding of Science, generally shorthanded as the Bodmer Report—making the case that scientists had a responsibility to communicate their work with the public. The report led to the foundation of the Committee on the Public Understanding of Science in Britain.

You could also argue that it led to the rise of the “science communicator,” and to science celebrities like Bill Nye and Neil DeGrasse Tyson. And to shows like Cosmos.

So it’s not so much that the science itself—as a process, as an authority structure—has changed in recent years, says Bruce Lewenstein, a professor of science history and communication at Cornell. “What's different,” instead, “is the pressure to make your findings public and to engage with the public.” And that pressure, notes Patrick McCray, a science historian at the University of California, Santa Barbara, can be a highly productive thing. Scientists and the institutions that incubate their work often aim, he points out, not just to explain scientific discoveries, but also to popularize science itself. Each press release—and each YouTube video, and each Facebook update, and each upload to arXiv—serves as an update about a discovery. And also about Discovery as a broad human goal.

The end result of all this, McCray says, are scientific institutions that are “much more diverse in serving a whole range of audiences than you would have had 100 years ago.”

Which brings us back, again, to the Big Bang. And to the particular channels that are introduced a new understanding of it, this week, to the public at large. What they may sacrifice in terms of top-down control—of research, of messaging, of the stories we tell about the world we live in—those institutions are trying to compensate for in terms of public participation and interest. They're buying popularity with a little bit of provisionality. They're attempting to Cosmos our sense of the cosmos. And they're finding, in the process, a new way to codify knowledge—one YouTube video at a time.

This article available online at:

http://www.theatlantic.com/technology/archive/2014/03/one-of-the-greatest-discoveries-in-the-history-of-science-hasnt-been-peer-reviewed-does-it-matter/284488/