Top

14-03-2024 | Ambulatory Anesthesia (G Joshi, Section Editor)

Report and Interpretation of Randomized Controlled Trials with Statistically Nonsignificant Results for Primary Outcomes in the Anesthesia Domain: A Systematic Review

Authors: Onishi Tatsuki, Yoshika Onishi

Published in: Current Anesthesiology Reports

Abstract

Purpose of Review

Although the Declaration of Helsinki emphasizes the importance of accurate and transparent research communication, authors’ discretion in interpreting outcomes may spin their results and force a favorable outlook. This study investigated the use of “spin” in reporting randomized controlled trials (RCTs) in the anesthesia domain.

Recent Findings

We searched the PubMed database for articles reporting RCTs in the anesthesia domain published over 5 years (2014–2018). We excluded articles on cost-effectiveness, diagnostic test accuracy, and non-English language reports. The spin characteristics with reference to the location, extent, and strategy were assessed. We screened 799 articles and included 211 eligible articles. Spin reporting was found in 86 articles (40.8%), and was identified in the Results section in 75 articles, the results synthesis in the Discussion section in 52 articles, and the Conclusion section in 66 articles. The most common spin location was the Results section, and the type of spin was focused on statistically significant within-group comparisons. The most common spin strategy was language use that implied a benefit.

Summary

Spin reporting was highly prevalent in RCT reports in the anesthesia domain, and abstracts may effectively reflect the spin reporting status in articles. We scrutinized instances of spin within RCTs that failed to yield statistically significant results; however, the potential existence of spin in studies reporting statistically significant results may also exist. This warrants critical evaluation and consideration, and the application of a fragility index may be indispensable for this purpose.

Stanley K. Design of randomized controlled trials. Circulation. 2007;115:1164–9. https://doi.org/10.1161/CIRCULATIONAHA.105.594945.CrossRefPubMed

Victora CG, Habicht JP, Bryce J. Evidence-based public health: moving beyond randomized trials. Am J Public Health. 2004;94:400–5. https://doi.org/10.2105/ajph.94.3.400.CrossRefPubMedPubMedCentral

Cokkinos DV, Pantos C. Myocardial protection in man–from research concept to clinical practice. Heart Fail Rev. 2007;12:345–62. https://doi.org/10.1007/s10741-007-9030-5.CrossRefPubMed

Bellomo R, Bagshaw SM. Evidence-based medicine: classifying the evidence from clinical trials–the need to consider other dimensions. Crit Care. 2006;10:232. https://doi.org/10.1186/cc5045.CrossRefPubMedPubMedCentral

Dickersin K, Straus SE, Bero LA. Evidence based medicine: increasing, not dictating, choice. BMJ. 2007;334:s10. https://doi.org/10.1136/bmj.39062.639444.94.CrossRefPubMed

Heneghan C, Mahtani KR, Goldacre B, Godlee F, Macdonald H, Jarvies D. Evidence based medicine manifesto for better healthcare. BMJ. 2017;357:j2973. https://doi.org/10.1136/bmj.j2973.CrossRefPubMed

Horwitz RI, Hayes-Conroy A, Caricchio R, Singer BH. From evidence based medicine to medicine based evidence. Am J Med. 2017;130:1246–50. https://doi.org/10.1016/j.amjmed.2017.06.012.CrossRefPubMed

Chapman G, Talbot N, McCartney D, Tippett V, Burch D. Evidence based medicine–older, but no better educated? Lancet. 2013;382:1484. https://doi.org/10.1016/S0140-6736(13)62286-2.CrossRefPubMed

Greenhalgh T, Howick J, Maskrey N, Evidence Based Medicine Renaissance Group. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725. https://doi.org/10.1136/bmj.g3725.CrossRefPubMedPubMedCentral

10.

Montori VM, Guyatt GH. Progress in evidence-based medicine. JAMA. 2008;300:1814–6. https://doi.org/10.1001/jama.300.15.1814.CrossRefPubMed

11.

Haynes RB. What kind of evidence is it that evidence-based medicine advocates want health care providers and consumers to pay attention to? BMC Health Serv Res. 2002;2:3. https://doi.org/10.1186/1472-6963-2-3.CrossRefPubMedPubMedCentral

12.

Declaration of Helsinki. https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. Accessed 7 April 2023.

13.

International clinical trials registry platform (ICTRP). https://www.who.int/clinical-trials-registry-platform. Accessed 7 April 2023.

14.

EU Clinical Trials Register. https://www.clinicaltrialsregister.eu/. Accessed 7 April 2023.

15.

Clinicaltrials.Gov. https://clinicaltrials.gov/. Accessed 7 April 2023.

16.

UMIN clinical trials registry (UMIN-CTR) https://www.umin.ac.jp/ctr/index-j.htm. Accessed 7 April 2023.

17.

Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637–9. https://doi.org/10.1001/jama.276.8.637.CrossRefPubMed

18.

•• Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically non-significant results for primary outcomes. JAMA. 2010;303:2058–64. https://doi.org/10.1001/jama.2010.651. Review that first mentioned the distorted interpretation of reporting in randomized controlled trials as spin.

19.

Gewandter JS, McKeown A, McDermott MP, et al. Data interpretation in analgesic clinical trials with statistically non-significant primary analyses: an ACTTION systematic review. J Pain. 2015;16:3–10. https://doi.org/10.1016/j.jpain.2014.10.003.CrossRefPubMed

20.

Latronico N, Metelli M, Turin M, Piva S, Rasulo FA, Minelli C. Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010. Intensive Care Med. 2013;39:1386–95. https://doi.org/10.1007/s00134-013-2947-3.CrossRefPubMed

21.

Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252–60. https://doi.org/10.1056/NEJMsa065779.CrossRefPubMed

22.

Turner EH, Cipriani A, Furukawa TA, Salanti G, de Vries YA. Selective publication of antidepressant trials and its influence on apparent efficacy: updated comparisons and meta-analyses of newer versus older trials. PLOS Med. 2022;19:e1003886. https://doi.org/10.1371/journal.pmed.1003886.CrossRefPubMedPubMedCentral

23.

You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30:210–6. https://doi.org/10.1200/JCO.2011.37.0890.CrossRefPubMed

24.

Arunachalam L, Hunter IA, Killeen S. Reporting of randomized controlled trials with statistically non-significant primary outcomes published in high-impact surgical journals. Ann Surg. 2017;265:1141–5. https://doi.org/10.1097/SLA.0000000000001795.CrossRefPubMed

25.

Hemming K, Javid I, Taljaard M. A review of high impact journals found that misinterpretation of non-statistically significant results from randomized trials was common. J Clin Epidemiol. 2022;145:112–20. https://doi.org/10.1016/j.jclinepi.2022.01.014.CrossRefPubMed

26.

• Roberts WB, Cooper CM, Khattab M, et al. Evaluation of ‘spin’ in the abstracts of randomized controlled trial reports in cardiology. J Am Osteopath Assoc. 2020. https://doi.org/10.7556/jaoa.2020.133. Review regarding spin in abstracts of cardiology RCTs as clinicians often base their decisions solely on abstracts.

27.

• Kinder NC, Weaver MD, Wayant C, Vassar M. Presence of ‘spin’ in the abstracts and titles of anaesthesiology randomised controlled trials. Br J Anaesth. 2019;122:e13–4. https://doi.org/10.1016/j.bja.2018.10.023. Review based on spin in abstracts and titles for they may misrepresent RCT findings.

28.

Grolleau F, Collins GS, Smarandache A, et al. The fragility and reliability of conclusions of anesthesia and critical care randomized trials with statistically significant findings: a systematic review. Crit Care Med. 2019;47:456–62. https://doi.org/10.1097/CCM.0000000000003527.CrossRefPubMed

29.

Demarquette A, Perrault T, Alapetite T, et al. Spin and fragility in randomised controlled trials in the anaesthesia literature: a systematic review. Br J Anaesth. 2023;130:528–35. https://doi.org/10.1016/j.bja.2023.01.001.CrossRefPubMed

30.

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLOS Med. 2009;6:e1000100. https://doi.org/10.1371/journal.pmed.1000100.CrossRefPubMedPubMedCentral

31.

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. https://doi.org/10.2307/2529310.CrossRefPubMed

32.

• Wasserstein RL, Lazar NA. The ASA statement on p-values: context, process, and purpose. Am Stat. 2016;70:129–33. https://doi.org/10.1080/00031305.2016.1154108. Statement to review the attitude of biasing p < 0.05.

33.

Baker M. Statisticians issue warning over misuse of P values. Nature. 2016;531:151. https://doi.org/10.1038/nature.2016.19503.ADSCrossRefPubMed

34.

Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias – an updated review. PLoS One. 2013;8:e66844. https://doi.org/10.1371/journal.pone.0066844.ADSCrossRefPubMedPubMedCentral

35.

Rennie D, Glass RM. Structuring abstracts to make them more informative. JAMA. 1991;266:116–7. https://doi.org/10.1001/jama.1991.03470010120043.CrossRefPubMed

36.

Eid T, vanSonnenberg E, Azar A, Mistry P, Eid K, Kang P. Analysis of the variability of abstract structures in medical journals. J Gen Intern Med. 2018;33:1013–4. https://doi.org/10.1007/s11606-018-4428-4.CrossRefPubMedPubMedCentral

37.

Ibrahim AM. Seeing is believing: using visual abstracts to disseminate scientific research. Am J Gastroenterol. 2018;113:459–61. https://doi.org/10.1038/ajg.2017.268.CrossRefPubMed

Title: Report and Interpretation of Randomized Controlled Trials with Statistically Nonsignificant Results for Primary Outcomes in the Anesthesia Domain: A Systematic Review
Authors: Onishi Tatsuki
Yoshika Onishi
Publication date: 14-03-2024
Publisher: Springer US
Published in: Current Anesthesiology Reports
Electronic ISSN: 2167-6275
DOI: https://doi.org/10.1007/s40140-024-00625-0

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Report and Interpretation of Randomized Controlled Trials with Statistically Nonsignificant Results for Primary Outcomes in the Anesthesia Domain: A Systematic Review

Abstract

Purpose of Review

Recent Findings

Summary

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose of Review

Recent Findings

Summary

Please log in to get access to this content