But according to a new study published in The BMJ, these symptom checkers vary widely in accuracy. Researchers from Harvard Medical School and Boston’s Children’s Hospital input 45 “patient vignettes”—some common, some uncommon conditions—into 23 different symptom checkers.
On average, they included the correct diagnosis in the first three results 51 percent of the time, and included it in the first 20 results 58 percent of the time (the individual checkers ranged from 34 percent to 84 percent). The chances of getting the right diagnosis listed first were really hit or miss. On the low end of the spectrum MEDoctor listed the correct condition first 5 percent of the time; on the high end, DocResponse listed it 50 percent of the time. Understandably, the sites did better at identifying common conditions than uncommon ones.
Of the symptom checkers that gave triage advice (whether the condition requires medical attention), they did so “appropriately,” according to the researchers, 57 percent of the time. Triage advice was more likely to be correct for more urgent conditions or uncommon conditions.
“Although there was a range of performance across symptom checkers, overall they had deficits in both diagnosis and triage accuracy,” the study reads. It’s also possible that the study over-estimated how accurate these sites are, because the patient vignettes it used included clinical terms (like “mouth ulcers,” for example) that a layperson might not necessarily know or recognize his symptoms as.
One of the other issues the researchers saw was that the sites weren’t very good, on the whole, at recommending self-care when it was appropriate, which could lead to people going to the ER or doctor’s office for something that would go away on its own, or that could be treated over the counter.
As the researchers point out, the central question here is what role symptom checkers play for people. A lot of the antipathy toward people researching symptoms online I think is predicated on the idea that this would replace a doctor’s visit. If that were true, it would be a bad thing.
“If symptom checkers are seen as a replacement for seeing a physician, they are likely an inferior alternative,” the study reads. “It is believed that physicians have a diagnostic accuracy rate of 85 [to] 90 percent. However, in-person physician visits might be the wrong comparison because patients are likely not using symptom checkers to obtain a definitive diagnosis but for quick and accessible guidance.”
Most people are hopefully savvy enough not to take a WebMD result as a diagnosis. They’re probably just trying to figure out how seriously to take their condition. In that case, it might be less important that symptom checkers spit out the right diagnosis (and including it among 19 incorrect diagnoses is not super helpful anyway), and more important that they tell the searcher whether her symptoms warrant a trip to the hospital.