Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Systematic Reviews
Published in: Systematic Reviews 1/2016

Open Access 01-12-2016 | Research

Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines

Authors: Shannon E Kelly, David Moher, Tammy J Clifford

Published in: Systematic Reviews | Issue 1/2016

Login to get access

Abstract

Background

Rapid reviews are an accelerated evidence synthesis approach intended to meet the timely needs of decision-makers in healthcare settings. Quality of conduct and reporting has been described in the rapid review literature; however, no formal assessment has been carried out using available instruments. The objective of this study was to explore compliance with conduct and reporting guidelines in rapid reviews published or posted online during 2013 and 2014.

Methods

We performed a comprehensive literature search for rapid reviews using multiple bibliographic databases (e.g. PubMed, MEDLINE, EMBASE, the Cochrane Library) through December 31, 2014. Grey literature was searched thoroughly, and health technology assessment agencies were surveyed to identify additional rapid review products. Candidate reviews were assessed for inclusion using pre-specified eligibility criteria. Detailed data was collected from the included reviews on study and reporting characteristics and variables significant to rapid reviews (e.g. nomenclature, definition). We evaluated the quality of conduct and reporting of included rapid reviews using the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklists. Compliance with each checklist item was examined, and the sum of adequately reported items was used to describe overall compliance. Rapid reviews were stratified to explore differences in compliance related to publication status. The association between compliance and time to completion or length of publication was explored through univariate regression.

Results

Sixty-six rapid reviews were included. There were heterogeneous nomenclature, research questions and approaches to rapid reviews. Compliance with AMSTAR and PRISMA checklists was poor. Published rapid reviews were compliant with individual PRISMA items more often than unpublished reviews, but no difference was seen in AMSTAR item compliance overall. There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.

Conclusions

Transparency and inadequate reporting are significant limitations of rapid reviews. Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete. Further research may be warranted to explore reporting and conduct guidelines specific to rapid reviews and how these guidelines may be applied across the spectrum of rapid review approaches.
Appendix
Available only for authorised users
Footnotes
1
Note: The rapid review characteristics presented in this study are free from selective reporting. The surplus of study and methodological characteristics could not be adequately addressed in a single manuscript and will be explored in a future manuscript.
 
Literature
1.
2.
3.
Harker J, Kleijnen J. What is a rapid review? A methodological exploration of rapid reviews in health technology assessments. Int J Evid Based Healthc. 2012;10(4):397–410.CrossRefPubMed
4.
5.
Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S, Facey K, Hailey D, Norderhaug I, Maddern G. Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment. Int J Technol Assess Health Care. 2008;24(2):133–9.PubMed
6.
Hartling L, Guise JM, Kato E, Anderson J, Aronson N, Belinson S, Berliner E, Dryden D, Featherstone R, Foisy M, Mitchell M, Motu’apuaka M, Noorani H, Paynter R, Robinson KA, Schoelles K, Umscheid CA, Whitlock E. AHRQ comparative effectiveness reviews. EPC methods: an exploration of methods and context for the production of rapid reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2015.
7.
8.
9.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6, e1000097.CrossRefPubMedPubMedCentral
10.
Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10):1013–20.CrossRefPubMed
11.
Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw J. PRESS: Peer Review of Electronic Search Strategies. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2008.
12.
Banbury A, Roots A, Nancarrow S. Rapid review of applications of e-health and remote monitoring for rural residents. Aust J Rural Health. 2014;22(5):211–22.CrossRefPubMed
13.
Beall RF, Baskerville N, Golfam M, Saeed S, Little J. Modes of delivery in preventive intervention studies: a rapid review. Eur J Clin Invest. 2014;44(7):688–96.CrossRefPubMed
14.
Brunton G, Paraskeva N, Caird J, Bird KS, Kavanagh J, Kwan I, Stansfield C, Rumsey N, Thomas J. Psychosocial predictors, assessment, and outcomes of cosmetic procedures: a systematic rapid evidence assessment. Aesthetic Plast Surg. 2014;38(5):1030–40.CrossRefPubMed
15.
Bungay H, Vella-Burrows T. The effects of participating in creative activities on the health and well-being of children and young people: a rapid review of the literature. Perspect Public Health. 2013;133(1):44–52.CrossRefPubMed
16.
Chaiyachati KH, Ogbuoji O, Price M, Suthar AB, Negussie EK, Barnighausen T. Interventions to improve adherence to antiretroviral therapy: a rapid systematic review. Aids. 2014;28 Suppl 2:S187–204.CrossRefPubMed
17.
Cooper CL, Hind D, Duncan R, Walters S, Lartey A, Lee E, Bradburn M. A rapid review indicated higher recruitment rates in treatment trials than in prevention trials. J Clin Epidemiol. 2015;68(3):347–54.CrossRefPubMed
18.
Costello RB, Lentino CV, Boyd CC, O’Connell ML, Crawford CC, Sprengel ML, Deuster PA. The effectiveness of melatonin for promoting healthy sleep: a rapid evidence assessment of the literature. Nutr J. 2014;13:106.CrossRefPubMedPubMedCentral
19.
Delgado R, York A, Lee C, Crawford C, Buckenmaier III C, Schoomaker E, Crawford P. Assessing the quality, efficacy, and effectiveness of the current evidence base of active self-care complementary and integrative medicine therapies for the management of chronic pain: a rapid evidence assessment of the literature. Pain Med. 2014;15(1):S9–S20.CrossRefPubMed
20.
Dennis SM, Harris M, Lloyd J, Powell Davies G, Faruqi N, Zwar N. Do people with existing chronic conditions benefit from telephone coaching? A rapid review. Aust Health Rev. 2013;37(3):381–8.PubMed
21.
Dorresteijn PM, Ipenburg NA, Murphy KJ, Smit M, Van Vulpen JK, Wegner I, Stegeman I, Grolman W. Rapid systematic review of normal audiometry results as a predictor for benign paroxysmal positional vertigo. Otolaryngol Head Neck Surg. 2014;150(6):919–24.CrossRefPubMed
22.
Garrett B, Taverner T, Masinde W, Gromala D, Shaw C, Negraeff M. A rapid evidence assessment of immersive virtual reality as an adjunct therapy in acute pain management in clinical practice. Clin J Pain. 2014;30(12):1089–98.CrossRefPubMed
23.
Hithersay R, Strydom A, Moulster G, Buszewicz M. Carer-led health interventions to monitor, promote and improve the health of adults with intellectual disabilities in the community: a systematic review. Res Dev Disabil. 2014;35(4):887–907.CrossRefPubMed
24.
Jarrett N, Scott I, Addington-Hall J, Amir Z, Brearley S, Hodges L, Richardson A, Sharpe M, Stamataki Z, Stark D, Siller C, Ziegler L, Foster C. Informing future research priorities into the psychological and social problems faced by cancer survivors: a rapid review and synthesis of the literature. Eur J Oncol Nurs. 2013;17(5):510–20.CrossRefPubMed
25.
Khorsan R, Crawford C, Ives JA, Walter AR, Jonas WB. The effect of omega-3 fatty acids on biomarkers of inflammation: a rapid evidence assessment of the literature. Mil Med. 2014;179(11 Suppl):2–60.CrossRefPubMed
26.
Kim DG, Choi YY, An JY, Kwon IG, Cho I, Kim YM, Bae JM, Song MG, Noh SH. Comparing the short-term outcomes of totally intracorporeal gastroduodenostomy with extracorporeal gastroduodenostomy after laparoscopic distal gastrectomy for gastric cancer: a single surgeon’s experience and a rapid systematic review with meta-analysis. Surg Endosc. 2013;27(9):3153–61.CrossRefPubMed
27.
Lal S, Adair CE. E-mental health: a rapid review of the literature. Psychiatr Serv. 2014;65(1):24–32.CrossRefPubMed
28.
Loveday HP, Wilson JA, Kerr K, Pitchers R, Walker JT, Browne J. Association between healthcare water systems and Pseudomonas aeruginosa infections: a rapid systematic review. J Hosp Infect. 2014;86(1):7–15.CrossRefPubMed
29.
Luckett T, Phillips J, Agar M, Virdun C, Green A, Davidson PM. Elements of effective palliative care models: a rapid review. BMC Health Serv Res. 2014;14:136.CrossRefPubMedPubMedCentral
30.
McLean SL, Blenkinsopp A, Bennett MI. Using haloperidol as an antiemetic in palliative care: informing practice through evidence from cancer treatment and postoperative contexts. J Pain Palliat Care Pharmacother. 2013;27(2):132–5.CrossRefPubMed
31.
Menear M, Briand C. Implementing a continuum of evidence-based psychosocial interventions for people with severe mental illness: part 1-review of major initiatives and implementation strategies. Can J Psychiatry. 2014;59(4):178–86.PubMedPubMedCentral
32.
Moe-Byrne T, Chambers D, Harden M, McDaid C. Behaviour change interventions to promote prescribing of generic drugs: a rapid evidence synthesis and systematic review. BMJ Open. 2014;4(5), e004623.CrossRefPubMedPubMedCentral
33.
34.
Paton F, Chambers D, Wilson P, Eastwood A, Craig D, Fox D, Jayne D, McGinnes E. Effectiveness and implementation of enhanced recovery after surgery programmes: a rapid evidence synthesis. BMJ Open. 2014;4(7), e005015.CrossRefPubMedPubMedCentral
35.
Reinink H, Wegner I, Stegeman I, Grolman W. Rapid systematic review of repeated application of the epley maneuver for treating posterior BPPV. Otolaryngol Head Neck Surg. 2014;151(3):399–406.CrossRefPubMed
36.
Saeed S, Golfam M, Beall RF, Ashbury FD, Palmer LJ, Little J. Effectiveness of individual-focused interventions to prevent chronic disease. Eur J Clin Invest. 2014;44(9):883–91.CrossRefPubMed
37.
Treanor CJ, Donnelly M. The late effects of cancer and cancer treatment: a rapid review. J Community Support Oncol. 2014;12(4):137–48.PubMed
38.
Trivedy CR, Cooke MW. Unscheduled return visits (URV) in adults to the emergency department (ED): a rapid evidence assessment policy review. Emerg Med J. 2013;28.
39.
van der Scheer-Horst ES, Van Benthem PP, Bruintjes TD, Van Leeuwen RB, van der Zaag-Loonen HJ. The efficacy of vestibular rehabilitation in patients with benign paroxysmal positional vertigo: a rapid review. Otolaryngol Head Neck Surg. 2014;151(5):740–5.CrossRefPubMed
40.
van der Veen EL, Van Hulst RA, De Ru JA. Hyperbaric oxygen therapy in acute acoustic trauma: a rapid systematic review. Otolaryngol Head Neck Surg. 2014;151(1):42–5.CrossRefPubMed
41.
Van Duijn JG, Isfordink LM, Nij Bijvank JA, Stapper CW, Van Vuren AJ, Wegner I, Kortekaas MF, Grolman W. Rapid systematic review of the epley maneuver for treating posterior canal benign paroxysmal positional vertigo. Otolaryngol Head Neck Surg. 2014;150(6):925–32.CrossRefPubMed
42.
Wegner I, Niesten ME, Van Werkhoven CH, Grolman W. Rapid systematic review of the epley maneuver versus vestibular rehabilitation for benign paroxysmal positional vertigo. Otolaryngol Head Neck Surg. 2014;151(2):201–7.CrossRefPubMed
43.
Wolfenden L, Carruthers J, Wyse R, Yoong S. Translation of tobacco control programs in schools: findings from a rapid review of systematic reviews of implementation and dissemination interventions. Health Promot J Austr. 2014;25(2):136–8.CrossRefPubMed
44.
Zeno SA, Purvis D, Crawford C, Lee C, Lisman P, Deuster PA. Warm-ups for military fitness testing: rapid evidence assessment of the literature. Med Sci Sports Exerc. 2013;45(7):1369–76.CrossRefPubMed
45.
46.
47.
48.
49.
50.
51.
52.
53.
Department of Veterans Affairs, Evidence-based Synthesis Program (ESP). Evidence map of yoga for high-impact conditions affecting veterans. [Internet] Washington, USA: Department of Veterans Affairs, Veterans Health Administration, Quality Enhancement Research Initiative, Health Services Research and Development Service; 2014; Available from: http://​www.​hsrd.​research.​va.​gov/​publications/​esp/​yoga.​pdf.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
The Knowledge Synthesis (KS) Group, KS Canada, Ottawa Methods Centre, Ottawa Hospital Research Institute. Do outcomes vary among different curricula or models of delivery for pre-licensure nursing education? A rapid review. Ottawa. 2013.
73.
The Knowledge Synthesis Group and Cochrane Collaborators. Ottawa Methods Centre, Ottawa Hospital Research Institute. Effects of performing complex pediatric intracavitary (IC) surgical procedures in specialized versus non-specialized centers in high risk children: Cochrane response rapid review. The Cochrane Collaboration: Ottawa, Ontario, Canada; 2013.
74.
75.
76.
Ontario Drug Policy Research Network (ODPRN). Rapid review unit. Comparative safety and effectiveness of inhaled long-acting agents (corticosteroids, beta agonists) for chronic obstructive pulmonary disease (COPD): a rapid review and network meta-analysis. Toronto: ODPRN, St. Michaels Hospital; 2014.
77.
78.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6, e1000100.CrossRefPubMedPubMedCentral
79.
Crawford C, Lee C, Bingham J, for the Active Self-Care Therapies for Pain Working Group. Sensory art therapies for the self-management of chronic pain symptoms. Pain Med. 2014;15:S66–75.CrossRefPubMed
80.
Crawford C, Lee C, Buckenmaier C, Schoomaker E, Petri R, Jonas W, for the Active Self-Care Therapies for Pain Working Group. The current state of the science for active self-care complementary and integrative medicine therapies in the management of chronic pain symptoms: lessons learned, directions for the future. Pain Med. 2014;15:S104–13.CrossRefPubMed
81.
Crawford C, Lee C, Freilich D, for the Active Self-Care Therapies for Pain Working Group. Effectiveness of active self-care complementary and integrative medicine therapies: options for the management of chronic pain symptoms. Pain Med. 2014;15:S86–95.CrossRefPubMed
82.
Crawford C, Lee C, May T, for the Active Self-Care Therapies for Pain Working Group. Physically oriented therapies for the self-management of chronic pain symptoms. Pain Med. 2014;15:S54–65.CrossRefPubMed
83.
Lee C, Crawford C, Hickey A, for the Active Self-Care Therapies for Pain Working Group. Mind–body therapies for the self-management of chronic pain symptoms. Pain Med. 2014;15:S21–39.CrossRefPubMed
84.
Lee C, Crawford C, Schoomaker E, for the Active Self-Care Therapies for Pain Working Group. Movement therapies for the self-management of chronic pain symptoms. Pain Med. 2014;15:S40–53.CrossRefPubMed
85.
Lee C, Crawford C, Swann S, for the Active Self-Care Therapies for Pain Working Group. Multimodal, integrative therapies for the self-management of chronic pain symptoms. Pain Med. 2014;15:S76–85.CrossRefPubMed
86.
Lee C, Crawford C, Teo L, Spevak C, for the Active Self-Care Therapies for Pain Working Group. An analysis of the various chronic pain conditions captured in a systematic review of active self-care complementary and integrative medicine therapies for the management of chronic pain symptoms. Pain Med. 2014;15:S96–S103.CrossRefPubMed
87.
Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S,Michie S, Moher D, Wager E. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76.CrossRefPubMed
88.
Windsor B, Popovich I, Jordan V, Showell M, Shea B, Farquhar C. Methodological quality of systematic reviews in subfertility: a comparison of Cochrane and non-Cochrane systematic reviews in assisted reproductive technologies. Hum Reprod. 2012;27(12):3460–6.CrossRefPubMed
89.
Adie S, Ma D, Harris IA, Naylor JM, Craig JC. Quality of conduct and reporting of meta-analyses of surgical interventions. Ann Surg. 2015;261(4):685–94.CrossRefPubMed
90.
Papageorgiou SN, Papadopoulos MA, Athanasiou AE. Reporting characteristics of meta-analyses in orthodontics: methodological assessment and statistical recommendations. Eur J Orthod. 2014;36(1):74–85.CrossRefPubMed
91.
92. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0 [updated March 2011]: The Cochrane Collaboration; 2011. Available from www.cochrane-handbook.org.
92.
Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, Group, Prisma- P. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.CrossRefPubMed
93.
94.
95.
Martens PJ, Roos NP. When health services researchers and policy makers interact: tales from the tectonic plates. Healthcare Policy. 2005;1(1):72–84.PubMedPubMedCentral
96.
Kung J, Chiappelli F, Cajulis OO, Avezova R, Kossan G, Chew L, Maida CA. From systematic reviews to clinical recommendations for evidence-based health care: validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. Open Dent J. 2010;4:84–91.PubMedPubMedCentral
97.
Pieper D, Buechter RB, Li L, Prediger B, Eikermann M. Systematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties. J Clin Epidemiol. 2015;68(5):574–83.CrossRefPubMed
98.
Maddern G. Rapid reviews or full health technology assessments—what are the needs in decision making? Health Technol Asses Int Meeting 2008 Montreal Canada. 2008.
99.
Van de Velde S, De Buck E, Dieltjens T, Aertgeerts B. Medicinal use of potato-derived products: conclusions of a rapid versus full systematic review. Phytother Res. 2011;25(5):787–8.CrossRefPubMed
100.
Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S, Facey K, Hailey D, Norderhaug I, Maddern G. Rapid versus full systematic reviews: validity in clinical practice? ANZ J Surg. 2008;78(11):1037–40.CrossRefPubMed
101.
Whitlock EP, Lin JS, Chou R, Shekelle P, Robinson KA. Using existing systematic reviews in complex systematic reviews. Ann Intern Med. 2008;148(10):776–82.CrossRefPubMed
102.
Lang A, Edwards N, Fleiszer A. Empty systematic reviews: hidden perils and lessons learned. J Clin Epidemiol. 2007;60(6):595–7.CrossRefPubMed
103.
Tricco AC, Moher D, Chen MH, Daniel R. Factors predicting completion and time to publication of Cochrane reviews. Open Medicine. 2009;3(4):e210–4.PubMedPubMedCentral
Metadata
Title
Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines
Authors
Shannon E Kelly
David Moher
Tammy J Clifford
Publication date
01-12-2016
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2016
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/s13643-016-0258-9

Other articles of this Issue 1/2016

Systematic Reviews 1/2016 Go to the issue

Reviewer acknowledgement

Thanks to all those who reviewed for Systematic Reviews in 2015

Protocol

Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial

Protocol

Pre-admission interventions to improve outcome after elective surgery—protocol for a systematic review

Protocol

Pain and disability following first-time lumbar fusion surgery for degenerative disorders: a systematic review protocol

Erratum

Erratum to: Describe the outcomes of dysvascular partial foot amputation and how these compare to transtibial amputation: a systematic review protocol for the development of shared decision making resources

Protocol

Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review protocol