Skip to main content
Top
Published in: Health and Quality of Life Outcomes 1/2012

Open Access 01-12-2012 | Commentary

Discussing study limitations in reports of biomedical studies- the need for more transparency

Authors: Milo A Puhan, Elie A Akl, Dianne Bryant, Feng Xie, Giovanni Apolone, Gerben ter Riet

Published in: Health and Quality of Life Outcomes | Issue 1/2012

Login to get access

Abstract

Unbiased and frank discussion of study limitations by authors represents a crucial part of the scientific discourse and progress. In today's culture of publishing many authors or scientific teams probably balance 'utter honesty' when discussing limitations of their research with the risk of being unable to publish their work. Currently, too few papers in the medical literature frankly discuss how limitations could have affected the study findings and interpretations. The goals of this commentary are to review how limitations are currently acknowledged in the medical literature, to discuss the implications of limitations in biomedical studies, and to make suggestions as to how to openly discuss limitations for scientists submitting their papers to journals. This commentary was developed through discussion and logical arguments by the authors who are doing research in the area of hedging (use of language to express uncertainty) and who have extensive experience as authors and editors of biomedical papers. We strongly encourage authors to report on all potentially important limitations that may have affected the quality and interpretation of the evidence being presented. This will not only benefit science but also offers incentives for authors: If not all important limitations are acknowledged readers and reviewers of scientific articles may perceive that the authors were unaware of them. Authors should take advantage of their content knowledge and familiarity with the study to prevent misinterpretations of the limitations by reviewers and readers. Articles discussing limitations help shape the future research agenda and are likely to be cited because they have informed the design and conduct of future studies. Instead of perceiving acknowledgment of limitations negatively, authors, reviewers and editors should recognize the potential of a frank and unbiased discussion of study limitations that should not jeopardize acceptance of manuscripts.
Literature
1.
go back to reference Feynman RP: Cargo Cult Science. Eng Sci 1974, 37(7):10–13. Feynman RP: Cargo Cult Science. Eng Sci 1974, 37(7):10–13.
2.
go back to reference Montori VM, Jaeschke R, Schunemann HJ, Bhandari M, Brozek JL, Devereaux PJ, Guyatt GH: Users' guide to detecting misleading claims in clinical research reports. BMJ 2004, 329(7474):1093–1096. 10.1136/bmj.329.7474.1093PubMedCentralCrossRefPubMed Montori VM, Jaeschke R, Schunemann HJ, Bhandari M, Brozek JL, Devereaux PJ, Guyatt GH: Users' guide to detecting misleading claims in clinical research reports. BMJ 2004, 329(7474):1093–1096. 10.1136/bmj.329.7474.1093PubMedCentralCrossRefPubMed
3.
go back to reference Ioannidis JP: Limitations are not properly acknowledged in the scientific literature. J Clin Epidemiol 2007, 60(4):324–329. 10.1016/j.jclinepi.2006.09.011CrossRefPubMed Ioannidis JP: Limitations are not properly acknowledged in the scientific literature. J Clin Epidemiol 2007, 60(4):324–329. 10.1016/j.jclinepi.2006.09.011CrossRefPubMed
4.
go back to reference Puhan MA, Heller N, Joleska I, Siebeling L, Muggensturm P, Umbehr M, Goodman S, ter Riet G: Acknowledging Limitations in Biomedical Studies: The ALIBI Study. In The Sixth International Congress on Peer Review and Biomedical Publication. Vancouver, Canada: JAMA and BMJ; 2009. Puhan MA, Heller N, Joleska I, Siebeling L, Muggensturm P, Umbehr M, Goodman S, ter Riet G: Acknowledging Limitations in Biomedical Studies: The ALIBI Study. In The Sixth International Congress on Peer Review and Biomedical Publication. Vancouver, Canada: JAMA and BMJ; 2009.
5.
go back to reference Hyland K: Hedging in scientific research articles. Amsterdam and Philadelphia: John Benjamins Publication Company; 1998.CrossRef Hyland K: Hedging in scientific research articles. Amsterdam and Philadelphia: John Benjamins Publication Company; 1998.CrossRef
6.
go back to reference Rothman KJ, Greenland S, Lash TL: Modern Epidemiology. 3rd edition. Lippincott Williams & Wilkins; 2008. Rothman KJ, Greenland S, Lash TL: Modern Epidemiology. 3rd edition. Lippincott Williams & Wilkins; 2008.
7.
go back to reference Clark S, Horton R: Putting research into context--revisited. Lancet 2010, 376(9734):10–11. 10.1016/S0140-6736(10)61001-XCrossRefPubMed Clark S, Horton R: Putting research into context--revisited. Lancet 2010, 376(9734):10–11. 10.1016/S0140-6736(10)61001-XCrossRefPubMed
8.
go back to reference Puhan MA, Guyatt GH, Goldstein R, Mador J, McKim D, Stahl E, Griffith L, Schunemann HJ: Relative responsiveness of the Chronic Respiratory Questionnaire, St. Georges Respiratory Questionnaire and four other health-related quality of life instruments for patients with chronic lung disease. Respir Med 2007, 101(2):308–316. 10.1016/j.rmed.2006.04.023CrossRefPubMed Puhan MA, Guyatt GH, Goldstein R, Mador J, McKim D, Stahl E, Griffith L, Schunemann HJ: Relative responsiveness of the Chronic Respiratory Questionnaire, St. Georges Respiratory Questionnaire and four other health-related quality of life instruments for patients with chronic lung disease. Respir Med 2007, 101(2):308–316. 10.1016/j.rmed.2006.04.023CrossRefPubMed
9.
go back to reference Teckle P, Peacock S, McTaggart-Cowan H, van der Hoek K, Chia S, Melosky B, Gelmon K: The ability of cancer-specific and generic preference-based instruments to discriminate across clinical and self-reported measures of cancer severities. Health Qual Life Outcomes 2011, 9: 106. 10.1186/1477-7525-9-106PubMedCentralCrossRefPubMed Teckle P, Peacock S, McTaggart-Cowan H, van der Hoek K, Chia S, Melosky B, Gelmon K: The ability of cancer-specific and generic preference-based instruments to discriminate across clinical and self-reported measures of cancer severities. Health Qual Life Outcomes 2011, 9: 106. 10.1186/1477-7525-9-106PubMedCentralCrossRefPubMed
10.
go back to reference Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ: What is "quality of evidence" and why is it important to clinicians? BMJ 2008, 336(7651):995–998. 10.1136/bmj.39490.551019.BEPubMedCentralCrossRefPubMed Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ: What is "quality of evidence" and why is it important to clinicians? BMJ 2008, 336(7651):995–998. 10.1136/bmj.39490.551019.BEPubMedCentralCrossRefPubMed
11.
go back to reference Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336(7650):924–926. 10.1136/bmj.39489.470347.ADPubMedCentralCrossRefPubMed Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336(7650):924–926. 10.1136/bmj.39489.470347.ADPubMedCentralCrossRefPubMed
12.
go back to reference Haynes RB, Sackett D, Tugwell P, Guyatt GH: Clinical Epidemiology: How to Do Clinical Practice Research. 3rd edition. Lippincott Williams & Wilkins; 2005. Haynes RB, Sackett D, Tugwell P, Guyatt GH: Clinical Epidemiology: How to Do Clinical Practice Research. 3rd edition. Lippincott Williams & Wilkins; 2005.
13.
go back to reference Bilotta C, Bowling A, Nicolini P, Case A, Pina G, Rossi SV, Vergani C: Older People's Quality of Life (OPQOL) scores and adverse health outcomes at a one-year follow-up. A prospective cohort study on older outpatients living in the community in Italy. Health Qual Life Outcomes 2011, 9: 72. 10.1186/1477-7525-9-72PubMedCentralCrossRefPubMed Bilotta C, Bowling A, Nicolini P, Case A, Pina G, Rossi SV, Vergani C: Older People's Quality of Life (OPQOL) scores and adverse health outcomes at a one-year follow-up. A prospective cohort study on older outpatients living in the community in Italy. Health Qual Life Outcomes 2011, 9: 72. 10.1186/1477-7525-9-72PubMedCentralCrossRefPubMed
14.
go back to reference Bausewein C, Simon ST, Benalia H, Downing J, Mwangi-Powell FN, Daveson BA, Harding R, Higginson IJ: Implementing patient reported outcome measures (PROMs) in palliative care--users' cry for help. Health Qual Life Outcomes 2011, 9: 27. 10.1186/1477-7525-9-27PubMedCentralCrossRefPubMed Bausewein C, Simon ST, Benalia H, Downing J, Mwangi-Powell FN, Daveson BA, Harding R, Higginson IJ: Implementing patient reported outcome measures (PROMs) in palliative care--users' cry for help. Health Qual Life Outcomes 2011, 9: 27. 10.1186/1477-7525-9-27PubMedCentralCrossRefPubMed
Metadata
Title
Discussing study limitations in reports of biomedical studies- the need for more transparency
Authors
Milo A Puhan
Elie A Akl
Dianne Bryant
Feng Xie
Giovanni Apolone
Gerben ter Riet
Publication date
01-12-2012
Publisher
BioMed Central
Published in
Health and Quality of Life Outcomes / Issue 1/2012
Electronic ISSN: 1477-7525
DOI
https://doi.org/10.1186/1477-7525-10-23

Other articles of this Issue 1/2012

Health and Quality of Life Outcomes 1/2012 Go to the issue