Nutrition Support Blog: Truthiness of Medical Literature
June 13, 2013
Statements and conclusions made on PubMed have a high degree of truthiness. Truthiness is a word popularized by comedian and political satirist Stephen Cobert, defined as the quality of assertions or claims that seem correct because they "feel right," regardless of evidence or facts. Unfortunately, there is a vast chasm packed with cold, hard facts that separates the realm of truthiness from that of truthfulness.
Many people are surprised to learn how often medical abstracts and articles fall short of truthfulness. It is natural to expect that scientific research and review articles that are subjected to the peer review and editorial process would be reliable sources of information. However, experts have decried the quality of study reporting and medical abstracts for years (1). In 1993 a diverse group of experts met to discuss study quality, and their meetings eventually resulted in the Consolidated Standards of Reporting Trials (CONSORT) Statement, first published in 1996, with the most recent version of guidelines published in 2010 (2). The guidelines offer a standardized way for investigators to report study findings:
Nevertheless, reviews of the medical literature document that inconsistencies and non-adherence to the CONSORT guidelines for abstracts, even among the high impact medical journals, continues to be quite common (3, 4). The degree of inaccuracy of medical abstracts can vary from minor variations from CONSORT guidelines to outright fabrications. A former co-worker once forwarded a PubMed abstract of an article that dealt with the fictitious “zombie epidemic”(5). The author had written an entire review article in the Journal of Clinical Nursing detailing interventions such as infection control, wound care, pain relief, medication delivery and even nutrition support of patients that were “infected” with a fictional illness that turned them into zombies (5). http://www.ncbi.nlm.nih.gov/pubmed/22211862
I was simultaneously horrified that anyone could pass this article through the peer review process to be published as a real review article, and filled with admiration over such a clever and successful prank. Evidently, everything the author of this fictional piece wrote had such a high degree of truthiness, no one stopped to see if any of the material was actually truthful.
Admittedly, there is long history of fabricated medical articles being passed off as real. As I mentioned in an older blog (Sept. 9, 2011), the father of modern Medicine, Sir William Osler, was known to publish fanciful and somewhat risqué case reports under a pseudonym. I also must confess to a certain amount of jealousy over the successful publication of the zombie article. My own attempt several years ago to add a certain amount of whimsy to our e-journal club on April 1st, by posting an obviously humorous review of nutrition support for marshmallow peeps (total peepenteral nutrition) brought me to the brink of termination of my employment.
I would encourage everyone to read the zombie abstract, and if you have access to read the full text, and then consider the implications of what can be allowed to pass into the medical literature the next time you read a nutrition or medical abstract or article. Certainly, our experience with critical evaluation of the medical literature (as outlined on our e-journal club) has been that most of the questionable aspects of the medical literature is far more subtle than “nutrition support for zombies,” but not always less suspect.
“A smart man only believes half of what he hears, a wise man knows which half.” - Jeff Cooper
"What's life without whimsy?"
- Sheldon Cooper, Big Bang Theory, (Chuck Lorre, Bill Prady)
1. McAlister FA, Clark HD, van Walraven C, et al. The medical review article revisited: has the science improved? Ann Intern Med. 1999 Dec 21;131(12):947-51.
2. Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010 Jun 1;152(11):726-32.
3. Bolignano D, Mattace-Raso F, Torino C, et al. The quality of reporting in clinical research: the CONSORT and STROBE initiatives. Aging Clin Exp Res. 2013 Apr;25(1):9-15.
4. Ghimire S, Kyung E, Kang W, Kim E. Assessment of adherence to the CONSORT statement for quality of reports on randomized controlled trial abstracts from four high-impact general medical journals. Trials. 2012 Jun 7;13:77.
5. Stanley D. The nurses' role in the prevention of Solanum infection: dealing with a zombie epidemic. J Clin Nurs. 2012 Jun;21(11-12):1606-13.