Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Intensive Care Medicine
Published in: Intensive Care Medicine 8/2013

01-08-2013 | Original

Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010

Authors: Nicola Latronico, Marta Metelli, Maddalena Turin, Simone Piva, Frank A. Rasulo, Cosetta Minelli

Published in: Intensive Care Medicine | Issue 8/2013

Login to get access

Abstract

Purpose

To evaluate the methodological quality of randomized controlled trials (RCTs) published in Intensive Care Medicine from 2001 to 2010, and to compare it with a previous review of RCTs published from 1975 to 2000.

Methods

We assessed the quality of reporting of randomization, blinding and participant flow, both individually and combined within the Jadad scale, and compared them with findings from our previous review. For RCTs published from 2001 to 2010, we also evaluated the frequency of distorted finding presentation (spin) and inflated predicted treatment effect (delta inflation).

Results

In the 221 RCTs from 2001 to 2010, the sample size was significantly larger than in the older series, and there was a higher proportion of studies with negative findings. Reporting of the rationale for sample size estimation and allocation concealment increased significantly, but reporting of other important individual methodological components did not change substantially compared with the previous period and remained low. Among RCTs from 2001 to 2010, a spin strategy was used in 69 of 111 RCTs with statistically negative results, while delta inflation was present in 7 of 11 RCTs evaluating survival as a primary outcome. Papers with higher Jadad scores were cited more often than the others.

Conclusions

Quality of reporting of RCTs published in Intensive Care Medicine has only partly improved over time, and spin and delta bias are of frequent occurrence. There is a need for stronger adherence to CONSORT recommendations, with special emphasis on accurate description of randomization and blindness, and correct reporting of statistically non-significant results.
Appendix
Available only for authorised users
Literature
1.
Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S (1995) Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 16:62–73PubMedCrossRef
2.
Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273:408–412PubMedCrossRef
3.
Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352:609–613PubMedCrossRef
4.
Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gotzsche PC, Lang T (2001) The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134:663–694PubMedCrossRef
5.
Kjaergard LL, Villumsen J, Gluud C (2001) Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 135:982–989PubMedCrossRef
6.
Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S (2012) Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ 344:e1119PubMedCrossRef
7.
Savovic J, Jones HE, Altman DG, Harris RJ, Juni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, JP AI, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JA (2012) Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med
8.
Jüni P, Altman DG, Egger M (2008) Assessing the quality of randomised controlled trialssystematic reviews in health care. BMJ, London, pp 87–108
9.
Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17:1–12PubMedCrossRef
10.
11.
Huwiler-Muntener K, Juni P, Junker C, Egger M (2002) Quality of reporting of randomized trials as a measure of methodologic quality. JAMA 287:2801–2804PubMedCrossRef
12.
Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA (2008) Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336:601–605PubMedCrossRef
13.
Aberegg SK, Richards DR, O’Brien JM (2010) Delta inflation: a bias in the design of randomized controlled trials in critical care medicine. Crit Care 14:R77PubMedCrossRef
14.
Boutron I, Dutton S, Ravaud P, Altman DG (2010) Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 303:2058–2064PubMedCrossRef
15.
Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG (2012) Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA 308:2594–2604PubMedCrossRef
16.
Latronico N, Botteri M, Minelli C, Zanotti C, Bertolini G, Candiani A (2002) Quality of reporting of randomised controlled trials in the intensive care literature. A systematic analysis of papers published in Intensive Care Medicine over 26 years. Intensive Care Med 28:1316–1323PubMedCrossRef
17.
Azoulay E, Citerio G, Timsit JF (2013) The identity of Intensive Care Medicine. Intensive Care Med 39:343–344CrossRef
18.
Sud S, Sud M, Friedrich JO, Meade MO, Ferguson ND, Wunsch H, Adhikari NK (2010) High frequency oscillation in patients with acute lung injury and acute respiratory distress syndrome (ARDS): systematic review and meta-analysis. BMJ 340:c2327PubMedCrossRef
19.
Felder TM, Palmer NR, Lal LS, Mullen PD (2011) What is the evidence for pharmaceutical patient assistance programs? A systematic review. J Health Care Poor Underserved 22:24–49PubMed
20.
Poolman RW, Struijs PA, Krips R, Sierevelt IN, Lutz KH, Bhandari M (2006) Does a “Level I Evidence” rating imply high quality of reporting in orthopaedic randomised controlled trials? BMC Med Res Methodol 6:44PubMedCrossRef
21.
Lai TY, Wong VW, Lam RF, Cheng AC, Lam DS, Leung GM (2007) Quality of reporting of key methodological items of randomized controlled trials in clinical ophthalmic journals. Ophthalmic Epidemiol 14:390–398PubMedCrossRef
22.
Sut N, Senocak M, Uysal O, Koksalan H (2008) Assessing the quality of randomized controlled trials from two leading cancer journals using the CONSORT statement. Hematol Oncol Stem Cell Ther 1:38–43PubMed
23.
Bai Y, Gao J, Zou DW, Li ZS (2009) Methodological reporting of randomized clinical trials in major gastroenterology and hepatology journals in 2006. Hepatology 49:2108–2112PubMedCrossRef
24.
Danilla S, Wasiak J, Searle S, Arriagada C, Pedreros C, Cleland H, Spinks A (2009) Methodological quality of randomised controlled trials in burns care. A systematic review. Burns 35:956–961PubMedCrossRef
25.
Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG (2010) The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ 340:c723PubMedCrossRef
26.
Strech D, Soltmann B, Weikert B, Bauer M, Pfennig A (2011) Quality of reporting of randomized controlled trials of pharmacologic treatment of bipolar disorders: a systematic review. J Clin Psychiatr 72:1214–1221CrossRef
27.
Agha RA, Camm CF, Edison E, Orgill DP (2012) The methodological quality of randomized controlled trials in plastic surgery needs improvement: a systematic review. J Plast Reconstr Aesthet Surg
28.
Mills EJ, Wu P, Gagnier J, Devereaux PJ (2005) The quality of randomized trial reporting in leading medical journals since the revised CONSORT statement. Contemp Clin Trials 26:480–487PubMedCrossRef
29.
Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, Dias S, Schulz KF, Plint AC, Moher D (2012) Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev 11:MR000030
30.
Schulz KF, Altman DG, Moher D (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med 152:726–732PubMedCrossRef
31.
Dechartres A, Charles P, Hopewell S, Ravaud P, Altman DG (2011) Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed. J Clin Epidemiol 64:136–144PubMedCrossRef
32.
Hirst A, Altman DG (2012) Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PLoS ONE 7:e35621PubMedCrossRef
33.
Karanicolas PJ, Farrokhyar F, Bhandari M (2010) Practical tips for surgical research: blinding: who, what, when, why, how? Can J Surg 53:345–348PubMed
34.
Schulz KF, Altman DG, Moher D, Fergusson D (2010) CONSORT 2010 changes and testing blindness in RCTs. Lancet 375:1144–1146PubMedCrossRef
35.
Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P (2009) Reporting of sample size calculation in randomised controlled trials: review. BMJ 338:b1732PubMedCrossRef
36.
Moher D, Fortin P, Jadad AR, Juni P, Klassen T, Le Lorier J, Liberati A, Linde K, Penna A (1996) Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet 347:363–366PubMedCrossRef
37.
Schulz KF, Grimes DA (2002) Allocation concealment in randomised trials: defending against deciphering. Lancet 359:614–618PubMedCrossRef
38.
Guyatt GH, Mills EJ, Elbourne D (2008) In the era of systematic reviews, does the size of an individual trial still matter. PLoS Med 5:e4PubMedCrossRef
39.
Schulz KF, Grimes DA (2005) Multiplicity in randomised trials II: subgroup and interim analyses. Lancet 365:1657–1661PubMedCrossRef
40.
Schulz KF, Grimes DA (2005) Multiplicity in randomised trials I: endpoints and treatments. Lancet 365:1591–1595PubMedCrossRef
41.
Schulz KF, Grimes DA (2005) Sample size calculations in randomised trials: mandatory and mystical. Lancet 365:1348–1353PubMedCrossRef
42.
Lundh A, Gotzsche PC (2008) Recommendations by Cochrane Review Groups for assessment of the risk of bias in studies. BMC Med Res Methodol 8:22PubMedCrossRef
43.
44.
International Committee of Medical Journal Editors (2013) Uniform requirements for manuscripts submitted to biomedical journals: publishing and editorial issues related to publication in biomedical journals: obligation to publish negative studies. http://​www.​icmje.​org/​publishing_​1negative.​html. Accessed 2 April 2013
45.
46.
Powers JH (2008) Noninferiority and equivalence trials: deciphering ‘similarity’ of medical interventions. Stat Med 27:343–352PubMedCrossRef
47.
48.
Kesselheim AS, Robertson CT, Myers JA, Rose SL, Gillet VBA, Ross KM, Glynn RJ, Joffe S, Avorn J (2012) A randomized study of how physicians interpret research funding disclosures. N Engl J Med 367:1119–1127
49.
(2005) In praise of soft science. Nature 435:1003
50.
51.
Smith R (2003) Medical journals and pharmaceutical companies: uneasy bedfellows. BMJ 326:1202–1205PubMedCrossRef
52.
Smith R (2005) Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med 2:e138PubMedCrossRef
53.
Lundh A, Barbateskovic M, Hrobjartsson A, Gotzsche PC (2010) Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue—cohort study. PLoS Med 7:e1000354PubMedCrossRef
54.
McVeigh ME, Mann SJ (2009) The journal impact factor denominator: defining citable (counted) items. JAMA 302:1107–1109PubMedCrossRef
55.
Callaham M, Wears RL, Weber E (2002) Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA 287:2847–2850PubMedCrossRef
56.
Nieminen P, Carpenter J, Rucker G, Schumacher M (2006) The relationship between quality of research and citation frequency. BMC Med Res Methodol 6:42PubMedCrossRef
57.
Etter JF, Stapleton J (2009) Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 62:831–837PubMedCrossRef
58.
Filion KB, Pless IB (2008) Factors related to the frequency of citation of epidemiologic publications. Epidemiol Perspect Innov EP+I 5:3
Metadata
Title
Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010
Authors
Nicola Latronico
Marta Metelli
Maddalena Turin
Simone Piva
Frank A. Rasulo
Cosetta Minelli
Publication date
01-08-2013
Publisher
Springer Berlin Heidelberg
Published in
Intensive Care Medicine / Issue 8/2013
Print ISSN: 0342-4642
Electronic ISSN: 1432-1238
DOI
https://doi.org/10.1007/s00134-013-2947-3

Other articles of this Issue 8/2013

Intensive Care Medicine 8/2013 Go to the issue

Original

Epidemiology and outcome of thrombocytopenic patients in the intensive care unit: results of a prospective multicenter study

Correspondence

Diastolic dysfunction and N-terminal pro-brain natriuretic peptide in children with meningococcal sepsis

Imaging in Intensive Care Medicine

Where is the lead?

Systematic Review

Recommendations on the use of EEG monitoring in critically ill patients: consensus statement from the neurointensive care section of the ESICM

From the Inside

On being an intensivist

Correspondence

Precautions with heated humidifier systems in particular environments