Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Anesthesiology
Published in: BMC Anesthesiology 1/2012

Open Access 01-12-2012 | Research article

The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement

Authors: Victoria Borg Debono, Shiyuan Zhang, Chenglin Ye, James Paul, Aman Arya, Lindsay Hurlburt, Yamini Murthy, Lehana Thabane

Published in: BMC Anesthesiology | Issue 1/2012

Login to get access

Abstract

Background

Randomized controlled trials (RCTs) are routinely used in systematic reviews and meta-analyses that help inform healthcare and policy decision making. The proper reporting of RCTs is important because it acts as a proxy for health care providers and researchers to appraise the quality of the methodology, conduct and analysis of an RCT. The aims of this study are to analyse the overall quality of reporting in 23 RCTs that were used in a meta-analysis by assessing 3 key methodological items, and to determine factors associated with high quality of reporting. It is hypothesized that studies with larger sample sizes, that have funding reported, that are published in journals with a higher impact factor and that are in journals that have adopted or endorsed the CONSORT statement will be associated with better overall quality of reporting and reporting of key methodological items.

Methods

We systematically reviewed RCTs used within an anesthesiology related post-operative pain management meta-analysis. We included all of the 23 RCTs used, all of which were parallel design that addressed the use of femoral nerve block in improving outcomes after total knee arthroplasty. Data abstraction was done independently by two reviewers. The two main outcomes were: 1) 15 point overall quality of reporting score (OQRS) based on the Consolidated Standards for Reporting Trials (CONSORT) and 2) 3 point key methodological item score (KMIS) based on allocation concealment, blinding and intention-to-treat analysis.

Results

Twenty-three RCTs were included. The median OQRS was 9.0 (Interquartile Range = 3). A multivariable regression analysis did not show any significant association between OQRS or KMIS and our four predictor variables hypothesized to improve reporting. The direction and magnitude of our results when compared to similar studies suggest that the sample size and impact factor are associated with improved key methodological item reporting.

Conclusions

The quality of reporting of RCTs used within an anesthesia related meta-analysis is poor to moderate. The information gained from this study should be used by journals to register the urgency for RCTs to be clear and transparent in reporting to help make literature accessible and comparable.
Appendix
Available only for authorised users
Literature
1.
Schulz KF, Altman DG, Moher D, CONSORT Group: CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Med. 2010, 8: 18-10.1186/1741-7015-8-18.CrossRefPubMedPubMedCentral
2.
Rios LP, Ye C, Thabane L: Association between framing of the research question using the PICOT format and reporting quality of randomized controlled trials. BMC Med Res Methodol. 2010, 10: 11-10.1186/1471-2288-10-11.CrossRefPubMedPubMedCentral
3.
Rios LP, Odueyungbo A, Moitri MO, Rahman MO, Thabane L: Quality of reporting of randomized controlled trials in general endocrinology literature. J Clin Endocrinol Metab. 2008, 93 (10): 3810-3816. 10.1210/jc.2008-0817.CrossRefPubMed
4.
Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995, 273 (5): 408-412. 10.1001/jama.1995.03520290060030.CrossRefPubMed
5.
6.
Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG: CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010, 340: c869-10.1136/bmj.c869.CrossRefPubMedPubMedCentral
7.
Can OS, Yilmaz AA, Hasdogan M, Alkaya F, Turhan SC, Can MF, Alanoglu Z: Has the quality of abstracts for randomised controlled trials improved since the release of Consolidated Standards of Reporting Trial guideline for abstract reporting? A survey of four high-profile anaesthesia journals. Eur J Anaesthesiol. 2011, 28 (7): 485-492. 10.1097/EJA.0b013e32833fb96f.CrossRefPubMed
8.
Higgins JPT, Altman DG, Sterne JAC: Chapter 8 Assessing risk of bias in included studies. Cochrane Handbook for Systematic Reviews of Interventions. Edited by: Higgins JPT, Green S. 2011, The Cochrane Collaboration, , 8.0-510
9.
Juni P, Altman DG, Egger M: Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ. 2001, 323 (7303): 42-46. 10.1136/bmj.323.7303.42.CrossRefPubMedPubMedCentral
10.
Moher D, Jones A, Lepage L, CONSORT Group (Consolitdated Standards for Reporting of Trials): Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001, 285 (15): 1992-1995. 10.1001/jama.285.15.1992.CrossRefPubMed
11.
Moher D, Schulz KF, Altman DG: The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001, 357 (9263): 1191-1194. 10.1016/S0140-6736(00)04337-3.CrossRefPubMed
12.
Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP: Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?. Lancet. 1998, 352 (9128): 609-613. 10.1016/S0140-6736(98)01085-X.CrossRefPubMed
13.
Moher D, Altman DG, Schulz KF, Elbourne DR: Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond. CMAJ. 2004, 171 (4): 349-350. 10.1503/cmaj.1040031.CrossRefPubMedPubMedCentral
14.
Higgins JPT, Green S: Cochrane Collaboration: Cochrane handbook for systematic reviews of interventions. 2008, Wiley-Blackwell, Chichester, West Sussex; Hoboken NJCrossRef
15.
Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, Lau J: Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA. 2002, 287 (22): 2973-2982. 10.1001/jama.287.22.2973.CrossRefPubMed
16.
Kjaergard LL, Villumsen J, Gluud C: Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001, 135 (11): 982-989.CrossRefPubMed
17.
Chan AW, Altman DG: Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005, 365 (9465): 1159-1162. 10.1016/S0140-6736(05)71879-1.CrossRefPubMed
18.
Glasziou P, Meats E, Heneghan C, Shepperd S: What is missing from descriptions of treatment in trials and reviews?. BMJ. 2008, 336 (7659): 1472-1474. 10.1136/bmj.39590.732037.47.CrossRefPubMedPubMedCentral
19.
Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR: Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008, 3 (8): e3081-10.1371/journal.pone.0003081.CrossRefPubMedPubMedCentral
20.
Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG: The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ. 2010, 340: c723-10.1136/bmj.c723.CrossRefPubMedPubMedCentral
21.
Mills EJ, Wu P, Gagnier J, Devereaux PJ: The quality of randomized trial reporting in leading medical journals since the revised CONSORT statement. Contemp Clin Trials. 2005, 26 (4): 480-487. 10.1016/j.cct.2005.02.008.CrossRefPubMed
22.
Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C, Gaboury I: Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust. 2006, 185 (5): 263-267.PubMed
23.
Lai TY, Wong VW, Lam RF, Cheng AC, Lam DS, Leung GM: Quality of reporting of key methodological items of randomized controlled trials in clinical ophthalmic journals. Ophthalmic Epidemiol. 2007, 14 (6): 390-398. 10.1080/09286580701344399.CrossRefPubMed
24.
Mills E, Loke YK, Wu P, Montori VM, Perri D, Moher D, Guyatt G: Determining the reporting quality of RCTs in clinical pharmacology. Br J Clin Pharmacol. 2004, 58 (1): 61-65. 10.1111/j.1365-2125.2004.2092.x.CrossRefPubMedPubMedCentral
25.
Lai R, Chu R, Fraumeni M, Thabane L: Quality of randomized controlled trials reporting in the primary treatment of brain tumors. J Clin Oncol. 2006, 24 (7): 1136-1144. 10.1200/JCO.2005.03.1179.CrossRefPubMed
26.
Greenfield ML, Mhyre JM, Mashour GA, Blum JM, Yen EC, Rosenberg AL: Improvement in the quality of randomized controlled trials among general anesthesiology journals 2000 to 2006: a 6-year follow-up. Anesth Analg. 2009, 108 (6): 1916-1921. 10.1213/ane.0b013e31819fe6d7.CrossRefPubMed
27.
Greenfield ML, Rosenberg AL, O'Reilly M, Shanks AM, Sliwinski MJ, Nauss MD: The quality of randomized controlled trials in major anesthesiology journals. Anesth Analg. 2005, 100 (6): 1759-1764. 10.1213/01.ANE.0000150612.71007.A3.CrossRefPubMed
28.
Pua HL, Lerman J, Crawford MW, Wright JG: An evaluation of the quality of clinical trials in anesthesia. Anesthesiology. 2001, 95 (5): 1068-1073. 10.1097/00000542-200111000-00007.CrossRefPubMed
29.
Paul JE, Arya A, Hurlburt L, Cheng J, Thabane L, Tidy A, Murthy Y: Femoral nerve block improves analgesia outcomes after total knee arthroplasty: a meta-analysis of randomized controlled trials. Anesthesiology. 2010, 113 (5): 1144-1162. 10.1097/ALN.0b013e3181f4b18.CrossRefPubMed
30.
Chung KC, Kotsis SV: The ethics of clinical research. J Hand Surg Am. 2011, 36 (2): 308-315. 10.1016/j.jhsa.2010.11.012.CrossRefPubMed
31.
Nuesch E, Reichenbach S, Trelle S, Rutjes AW, Liewald K, Sterchi R, Altman DG, Juni P: The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study. Arthritis Rheum. 2009, 61 (12): 1633-1641. 10.1002/art.24894.CrossRefPubMed
32.
Nuesch E, Trelle S, Reichenbach S, Rutjes AW, Burgi E, Scherer M, Altman DG, Juni P: The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study. BMJ. 2009, 339: b3244-10.1136/bmj.b3244.CrossRefPubMedPubMedCentral
33.
Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008, 336 (7644): 601-605. 10.1136/bmj.39465.451748.AD.CrossRefPubMedPubMedCentral
34.
Schulz KF, Grimes DA, Altman DG, Hayes RJ: Blinding and exclusions after allocation in randomised controlled trials: survey of published parallel group trials in obstetrics and gynaecology. BMJ. 1996, 312 (7033): 742-744. 10.1136/bmj.312.7033.742.CrossRefPubMedPubMedCentral
35.
Huwiler-Muntener K, Juni P, Junker C, Egger M: Quality of reporting of randomized trials as a measure of methodologic quality. JAMA. 2002, 287 (21): 2801-2804. 10.1001/jama.287.21.2801.CrossRefPubMed
36.
Montori VM, Bhandari M, Devereaux PJ, Manns BJ, Ghali WA, Guyatt GH: In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol. 2002, 55 (8): 787-790. 10.1016/S0895-4356(02)00446-8.CrossRefPubMed
37.
Haahr MT, Hrobjartsson A: Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors. Clin Trials. 2006, 3 (4): 360-365.PubMed
38.
Miller LE, Stewart ME: The blind leading the blind: use and misuse of blinding in randomized controlled trials. Contemp Clin Trials. 2011, 32 (2): 240-243. 10.1016/j.cct.2010.11.004.CrossRefPubMed
39.
Newell DJ: Intention-to-treat analysis: implications for quantitative and qualitative research. Int J Epidemiol. 1992, 21 (5): 837-841. 10.1093/ije/21.5.837.CrossRefPubMed
40.
41.
Higgins JPT, Deeks JJ, Altman DG: Chapter 16: Special Topics in statistics. Cochrane Handbook for Systematic Reviews and Interventions. Edited by: Higgins JPT, Green S. 2011, The Cochrane Collaboration,, 16.0-16.2. http://​www.​cochrane-handbook.​org, 2011,
42.
Hollis S, Campbell F: What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999, 319 (7211): 670-674. 10.1136/bmj.319.7211.670.CrossRefPubMedPubMedCentral
43.
44.
Farrokhyar F, Chu R, Whitlock R, Thabane L: A systematic review of the quality of publications reporting coronary artery bypass grafting trials. Can J Surg. 2007, 50 (4): 266-277.PubMedPubMedCentral
45.
Thomas O, Thabane L, Douketis J, Chu R, Westfall AO, Allison DB: Industry funding and the reporting quality of large long-term weight loss trials. Int J Obes (Lond). 2008, 32 (10): 1531-1536. 10.1038/ijo.2008.137.CrossRef
46.
Balasubramanian SP, Wiener M, Alshameeri Z, Tiruvoipati R, Elbourne D, Reed MW: Standards of reporting of randomized controlled trials in general surgery: can we do better?. Ann Surg. 2006, 244 (5): 663-667. 10.1097/01.sla.0000217640.11224.05.CrossRefPubMedPubMedCentral
47.
Toulmonde M, Bellera C, Mathoulin-Pelissier S, Debled M, Bui B, Italiano A: Quality of Randomized Controlled Trials Reporting in the Treatment of Sarcomas. J Clin Oncol. 2011, 29 (9): 1204-1209. 10.1200/JCO.2010.30.9369.CrossRefPubMed
48.
Ivers NM, Taljaard M, Dixon S, Bennett C, McRae A, Taleban J, Skea Z, Brehaut JC, Boruch RF, Eccles MP, Grimshaw JM, Weijer C, Zwarenstein M, Donner A: Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000–8. BMJ. 2011, 343: d5886-10.1136/bmj.d5886.CrossRefPubMedPubMedCentral
49.
Eypasch E, Lefering R, Kum CK, Troidl H: Probability of adverse events that have not yet occurred: a statistical reminder. BMJ. 1995, 311 (7005): 619-620. 10.1136/bmj.311.7005.619.CrossRefPubMedPubMedCentral
50.
Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.CrossRefPubMed
51.
Egger M, Juni P, Bartlett C, CONSORT Group (Consolidated Standards of Reporting of Trials): Value of flow diagrams in reports of randomized controlled trials. JAMA. 2001, 285 (15): 1996-1999. 10.1001/jama.285.15.1996.CrossRefPubMed
52.
53.
Devereaux PJ, Choi PT, El-Dika S, Bhandari M, Montori VM, Schunemann HJ, Garg AX, Busse JW, Heels-Ansdell D, Ghali WA, Manns BJ, Guyatt GH: An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol. 2004, 57 (12): 1232-1236. 10.1016/j.jclinepi.2004.03.017.CrossRefPubMed
54.
Pildal J, Chan AW, Hrobjartsson A, Forfang E, Altman DG, Gotzsche PC: Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study. BMJ. 2005, 330 (7499): 1049-10.1136/bmj.38414.422650.8F.CrossRefPubMedPubMedCentral
55.
Hrobjartsson A, Pildal J, Chan AW, Haahr MT, Altman DG, Gotzsche PC: Reporting on blinding in trial protocols and corresponding publications was often inadequate but rarely contradictory. J Clin Epidemiol. 2009, 62 (9): 967-973. 10.1016/j.jclinepi.2009.04.003.CrossRefPubMed
56.
Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic B: Radiation Therapy Oncology Group: Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. BMJ. 2004, 328 (7430): 22-24. 10.1136/bmj.328.7430.22.CrossRefPubMedPubMedCentral
57.
Verhagen AP, de Vet HC, de Bie RA, Boers M, van den Brandt PA: The art of quality assessment of RCTs included in systematic reviews. J Clin Epidemiol. 2001, 54 (7): 651-654. 10.1016/S0895-4356(00)00360-7.CrossRefPubMed
58.
Olivo SA, Macedo LG, Gadotti IC, Fuentes J, Stanton T, Magee DJ: Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther. 2008, 88 (2): 156-175. 10.2522/ptj.20070147.CrossRefPubMed
59.
Juni P, Witschi A, Bloch R, Egger M: The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999, 282 (11): 1054-1060. 10.1001/jama.282.11.1054.CrossRefPubMed
60.
Herbison P, Hay-Smith J, Gillespie WJ: Adjustment of meta-analyses on the basis of quality scores should be abandoned. J Clin Epidemiol. 2006, 59 (12): 1249-1256.CrossRefPubMed
61.
Agha R, Cooper D, Muir G: The reporting quality of randomised controlled trials in surgery: A systematic review. Int J Surg. 2007, 5 (6): 413-422. 10.1016/j.ijsu.2007.06.002.CrossRefPubMed
62.
Reveiz L, Cortés-Jofré M, Asenjo Lobos C, Nicita G, Ciapponi A, Garcìa-Dieguez M, Tellez D, Delgado M, Solà I, Ospina E: Influence of trial registration on reporting quality of randomized trials: Study from highest ranked journals. J Clin Epidemiol. 2010, 63 (11): 1216-1222. 10.1016/j.jclinepi.2010.01.013.CrossRefPubMed
Metadata
Title
The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement
Authors
Victoria Borg Debono
Shiyuan Zhang
Chenglin Ye
James Paul
Aman Arya
Lindsay Hurlburt
Yamini Murthy
Lehana Thabane
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Anesthesiology / Issue 1/2012
Electronic ISSN: 1471-2253
DOI
https://doi.org/10.1186/1471-2253-12-13

Other articles of this Issue 1/2012

BMC Anesthesiology 1/2012 Go to the issue

Correspondence

Recent trends in publication of basic science and clinical research by United States investigators in anesthesia journals

Research article

Randomized comparison of the i-gel™, the LMA Supreme™, and the Laryngeal Tube Suction-D using clinical and fibreoptic assessments in elective patients

Study protocol

ASAP ECMO: Antibiotic, Sedative and Analgesic Pharmacokinetics during Extracorporeal Membrane Oxygenation: a multi-centre study to optimise drug therapy during ECMO

Research article

Theoretical effect of hyperventilation on speed of recovery and risk of rehypnotization following recovery - a GasMan®simulation

Research article

Early diagnostic markers of sepsis after oesophagectomy (including thromboelastography)

Research article

Use of a blood gas analyzer and a laboratory autoanalyzer in routine practice to measure electrolytes in intensive care unit patients