Impact Factor and the Future of Medical Journals

Some research publications are getting away from flawed measures of influence that make it easy to game the system.

Medical publishing, one of the oldest continuous scientific traditions, is growing exponentially. New medical journals dedicated to increasingly niche microcosms are being created every day. Yet medical publishing finds itself at a vital cross roads. This year's Nobel Prize winner in physiology, Randy Scheckman, announced his decision to boycott the three major “luxury” journals: Science, Nature, and Cell. Criticism of researchers' inability to replicate major medical findings is also gaining traction beyond scientific circles.

For an industry based on advancement, medical journals are very rigid. Take for example the impact factor, defined as the number of citations divided by the number of papers published in the journal, which is a measure to convey the influence of journals and the research they carry. Even in medical school, my friends would compare the impact factor of the journals they had published in, as though that represented the quality of the work. It is akin to what the “mega-pixel” is for digital cameras, a pithy misleading sound bite that has become an industry gold standard.

While the impact factor was originally devised in the 1970's by librarians to help them make rational choices about what journals to subscribe to, the current use is very different from that original intent. Impact factors are frequently considered for academic promotions, pressuring academics to keep targeting higher impact journals. In fact, scientists in China are paid based on the impact factor of the journals they publish in.

Journals employ several strategies to artificially raise the impact factor, which initiates a positive feedback loop by incentivizing more scientists to submit to them. Some editors have been caught trying to induce authors to increase the number of citations from their journal to further raise the impact factor. One investigation found that a collaboration of Brazilian journals had agreed to highly cite each other’s articles to fraudulently raise each of the journals’ impact factors.

Critics argue that citations are barely a reflection of the quality of the research and that the impact factor is easily manipulated by taking articles that are more likely to be highly cited or ‘sexy’ than highly meaningful. Reviews of common topics can contain hundreds of references (and authors have been known to cram them with self citations) and are more highly cited than original research articles that may eventually lead to a new discovery or finding. Also, research in controversial areas, such as a study published in Science that demonstrated brain damage by the drug Ecstasy, or the infamous paper published in the Lancet which purported to show a link between vaccines and autism, are more likely to be highly cited because of interest generated in these subjects (both studies were subsequently retracted).

Medical publishing’s growth is actually one of its greatest challenges. With more than half a million papers published annually in biomedicine alone, it is no wonder that physicians are increasingly feeling overwhelmed with the avalanche of information they find themselves facing. The new hubs of publishing are India and China, where new journals are being formed almost daily. My inbox usually receives several invitations a day to join editorial boards of journals I've never heard of, completely unrelated to my specialty.

And all of these journals need to be concerned with maintaining accuracy. The current system of peer-review, which originated in the 18th century, is now stressed. After a journal decides to consider a paper, it is sent to unpaid, anonymous reviewers in the same specialty to review. Physicians are busy and it's difficult for them to be able to effectively analyze a paper for accuracy. The only real upside for the reviewer is the synthetic prestige that comes with being an anonymous reviewer, which may not be enough to muster a consistent and thorough effort. The flaws of the current system were exposed in a sting by a reporter for Science who had a fraudulent and comically bad paper, from a fake alias and institution, accepted in the vast majority of journals he had submitted to without any meaningful review.

In spite of their strong ties to tradition, journal editors are finally responding to the changing ecology. When I asked Howard Bauchner, editor-in-chief of the Journal of the American Medical Association, what the future of the medical journal was, he summed it up in just one word: “Digital.” With a website curated with author video interviews, twitter feeds and podcasts, JAMA addresses one of the medium’s biggest drawbacks—passivity. “Medical journals have traditionally been passive; readers read articles and experts go on to discuss them. The digital format opens space for debate that journals haven’t previously explored,” Bauchner told me.

Presented by

Haider Javed Warraich is a staff physician at the Beth Israel Deaconess Medical Center in Boston.

How to Cook Spaghetti Squash (and Why)

Cooking for yourself is one of the surest ways to eat well. Bestselling author Mark Bittman teaches James Hamblin the recipe that everyone is Googling.

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register.

blog comments powered by Disqus


How to Cook Spaghetti Squash (and Why)

Cooking for yourself is one of the surest ways to eat well.


Before Tinder, a Tree

Looking for your soulmate? Write a letter to the "Bridegroom's Oak" in Germany.


The Health Benefits of Going Outside

People spend too much time indoors. One solution: ecotherapy.


Where High Tech Meets the 1950s

Why did Green Bank, West Virginia, ban wireless signals? For science.


Yes, Quidditch Is Real

How J.K. Rowling's magical sport spread from Hogwarts to college campuses


Would You Live in a Treehouse?

A treehouse can be an ideal office space, vacation rental, and way of reconnecting with your youth.

More in Health

Just In