The doctors of Wikiproject Medicine, an effort within Wikipedia to improve the quality of medical information by getting physicians and health care experts to edit the articles, suggest a different gold standard source.
“We try to use secondary sources like reviews, meta-analyses, and major textbooks,” says Dr. James Heilman, a clinical instructor in the University of British Columbia’s department of emergency medicine, and one of the most active medical editors on Wikipedia. His reasoning is that, for these secondary sources, someone has typically taken the time to review all of the peer-reviewed literature, which is often contradictory, and come to a conclusion based on what’s out there.
This study was not popular on the Wikiproject Medicine Talk page, with one commenter calling it “utterly meaningless and [not] worth the electrons it's printed on.”
Heilman, along with Dr. Samir Grover at the University of Toronto, is designing a study that he says would be a better way to test Wikipedia’s accuracy: have medical students take a standardized test using either medical textbooks or Wikipedia. Or, he says, you could look at whether an article’s statements are supported by the references it provides for those statements—not just any peer-reviewed source you find.
“Just because a reference is peer-reviewed doesn’t mean it’s a high-quality reference,” he says.
Wikipedia has its own peer review process before articles can be classified as “good” or “featured.” Heilman, who has participated in that process before, says “less than 1 percent” of Wikipedia’s medical articles have passed.
So both sides acknowledge: There are errors in Wikipedia’s health articles. And that’s a problem, because people use them.
“Amongst my colleagues, [using Wikipedia] is a little bit like flatulence,” Hasty says. “Everybody does it but nobody wants to admit to it…Whatever we can do to encourage one of the most popular sites to be more accurate is very good. I think that’s one of the positive things about this article.”
Both Hasty and Heilman emphasize that even as Wikipedia’s quality improves, no one making a serious medical decision—patient or doctor—should ever use just one source.
“It takes judgment, no matter what happens,” Hasty says. “That’s another reason we should use professionals who do devote their lives to ensuring patients get the best recommendations.”
Keeping that in mind, Wikiproject Medicine’s goal of having those same professionals editing Wikipedia pages seems like the best option. In the face of constantly-evolving medical literature, and primary—and even sometimes secondary—sources that contradict one another, professional judgement may be the goldest standard we have.