Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2017

Open Access 01-12-2017 | Research

Updated clinical guidelines experience major reporting limitations

Authors: Robin W.M. Vernooij, Laura Martínez García, Ivan Dario Florez, Laura Hildago Armas, Michiel H.F. Poorthuis, Melissa Brouwers, Pablo Alonso-Coello

Published in: Implementation Science | Issue 1/2017

Login to get access

Abstract

Background

The Checklist for the Reporting of Updated Guidelines (CheckUp) was recently developed. However, so far, no systematic assessment of the reporting of updated clinical guidelines (CGs) exists. We aimed to examine (1) the completeness of reporting the updating process in CGs and (2) the inter-observer reliability of CheckUp.

Methods

We conducted a systematic assessment of the reporting of the updating process in a sample of updated CGs using CheckUp. We performed a systematic search to identify updated CGs published in 2015, developed by a professional society, reporting a systematic review of the evidence, and containing at least one recommendation. Three reviewers independently assessed the CGs with CheckUp (16 items). We calculated the median score per item, per domain, and overall, converting scores to a 10-point scale. Multiple linear regression analyses were used to identify differences according to country, type of organisation, scope, and health topic of updated CGs. We calculated the intraclass coefficient (ICC) and 95% confidence interval (95% CI) for domains and overall score.

Results

We included in total 60 updated CGs. The median domain score on a 10-point scale for presentation was 5.8 (range 1.7 to 10), for editorial independence 8.3 (range 3.3 to 10), and for methodology 5.7 (range 0 to 10). The median overall score on a 10-point scale was 6.3 (range 3.1 to 10). Presentation and justification items at recommendation level (respectively reported by 27 and 38% of the CGs) and the methods used for the external review and implementing changes in practice were particularly poorly reported (both reported by 38% of the CGs). CGs developed by a European or international institution obtained a statistically significant higher overall score compared to North American or Asian institutions (p = 0.014). Finally, the agreement among the reviewers on the overall score was excellent (ICC 0.88, 95% CI 0.75 to 0.95).

Conclusions

The reporting of updated CGs varies considerably with significant room for improvement. We recommend using CheckUp to assess the updating process in updated CGs and as a blueprint to inform methods and reporting strategies in updating.
Appendix
Available only for authorised users
Literature
1.
Laine C, Taichman DB, Mulrow C. Trustworthy clinical guidelines. Ann Intern Med. 2011;154(11):774–5.CrossRefPubMed
2.
3.
4.
Martínez García L, Sanabria AJ, García Alvarez E, Trujillo-Martín MM, Etxeandia-Ikobaltzeta I, Kotzeva A, et al. The validity of recommendations from clinical guidelines: a survival analysis. CMAJ. 2014;186(16):1211–9.CrossRefPubMed
5.
Alderson LJ, Alderson P, Tan T. Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months. J Clin Epidemiol. 2014;67(1):52–5.CrossRefPubMed
6.
Neuman MD, Goldstein JN, Cirullo MA, Schwartz JS. Durability of class I American College of Cardiology/American Heart Association clinical practice guideline recommendations. JAMA. 2014;311(20):2092–100.CrossRefPubMedPubMedCentral
7.
Lyratzopoulos G, Barnes S, Stegenga H, Peden S, Campbell B. Updating clinical practice recommendations: is it worthwhile and when? Int J Technol Assess Health Care. 2012;28(1):29–35.CrossRefPubMed
8.
Shekelle PG, Ortiz E, Rhodes S, Morton SC, Eccles MP, Grimshaw JM, et al. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA. 2001;286(12):1461–7.CrossRefPubMed
9.
Martínez García L, Pardo-Hernandez H, McFarlane E, Sanabria AJ, Sparrow K, Alonso-Coello P, the G-I-N updating guidelines working group OBO. UpGlossary: Guidance on terminology and definitions for updating clinical guidelines. Cape Town: Global Evidence Summit 2017.
10.
Agbassi C, Messersmith H, McNair S, Brouwers M. Priority-based initiative for updating existing evidence-based clinical practice guidelines: the results of two iterations. J Clin Epidemiol. 2014;67(12):1335–42.CrossRefPubMed
11.
Martínez García L, Pardo H, Superchi C, Niño de Guzman E, Ballesteros M, Ibargoyen Roteta N, et al. Updating health decision-making tools: a systematic review of prioritisation processes. J Clin Epidemiol. 2017;86:11–24.CrossRefPubMed
12.
Martinez Garcia L, Sanabria AJ, Araya I, Lawson J, Solà I, Vernooij RW, et al. Efficiency of pragmatic search strategies to update clinical guidelines recommendations. BMC Med Res Methodol. 2015;15:57.CrossRefPubMedPubMedCentral
13.
Gartlehner G, West SL, Lohr KN, Kahwati L, Johnson JG, Harris RP, et al. Assessing the need to update prevention guidelines: a comparison of two methods. Int J Qual Health Care. 2004;16(5):399–406.CrossRefPubMed
14.
Johnston ME, Brouwers MC, Browman GP. Keeping cancer guidelines current: results of a comprehensive prospective literature monitoring strategy for twenty clinical practice guidelines. Int J Technol Assess Health Care. 2003;19(4):646–55.CrossRefPubMed
15.
Eccles M, Rousseau N, Freemantle N. Updating evidence-based clinical guidelines. J Health Serv Res Policy. 2002;7(2):98–103.CrossRefPubMed
16.
Parmelli E, Papini D, Moja L, Bandieri E, Belfiglio M, Ciccone G, et al. Updating clinical recommendations for breast, colorectal and lung cancer treatments: an opportunity to improve methodology and clinical relevance. Ann Oncol. 2011;22(1):188–94.CrossRefPubMed
17.
Martinez Garcia L, McFarlane E, Barnes S, Sanabria AJ, Alonso-Coello P, Alderson P. Updated recommendations: an assessment of NICE clinical guidelines. Implement Sci. 2014;9:72.CrossRefPubMedPubMedCentral
18.
Martinez Garcia L, Arevalo-Rodriguez I, Sola I, Haynes RB, Vandvik PO, Alonso-Coello P, et al. Strategies for monitoring and updating clinical practice guidelines: a systematic review. Implement Sci. 7:109.
19.
Vernooij RW, Sanabria AJ, Sola I, Alonso-Coello P, Martínez GL. Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implement Sci. 2014;9:3.CrossRefPubMedPubMedCentral
20.
Vernooij RW, Alonso-Coello P, Brouwers M, CheckUp Panel. Reporting items for updated clinical guidelines: checklist for the reporting of updated guidelines (CheckUp). PLoS medicine. 2017;14: e1002207.
21.
Grupo de trabajo para la actualización del Manual de Elaboración de GPC. Elaboración de Guías de Práctica Clínica en el Sistema Nacional de Salud. Actualización del Manual Metodológico. Madrid: Ministerio de Sanidad, Servicios Sociales e Igualdad - GuíaSalud; 2016.
22.
Eady EA, Layton AM, Sprakel J, BWM A, Fedorowicz Z, van Zuuren EJ. AGREE II assessments of recent acne treatment guidelines: how well do they reveal trustworthiness as defined by the Institute of Medicine (IOM) criteria? Br J Dermatol. 2017. doi:10.​1111/​bjd.​15777.
23.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.CrossRefPubMedPubMedCentral
24.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.CrossRefPubMed
25.
American Academy of Dental Sleep Medicine/American Academy of Sleep Medicine. Clinical practice guideline for the treatment of obstructive sleep apnea and snoring with oral appliance therapy: an update for 2015:2015.
26.
American Academy of Orthopaedic Surgeons. American Academy of Orthopaedic Surgeons clinical practice guideline on the treatment of pediatric diaphyseal femur fracture; 2015.
27.
American Academy of Otolaryngology–Head and Neck Surgery. Clinical practice guideline (update): adult sinusitis; 2015.
28.
American Academy of Sleep Medicine. Clinical practice guideline for the treatment of intrinsic circadian rhythm sleep-wake disorders: advanced sleep-wake phase disorder (ASWPD), delayed sleep-wake phase disorder (DSWPD), non-24-hour sleep-wake rhythm disorder (N24SWD), and irregular sleep-wake rhythm disorder (ISWRD). An update for 2015; 2015.
29.
American Cancer Society. Breast cancer screening for women at average risk: 2015 guideline update from the American Cancer Society; 2015.
30.
American College of Chest Physicians. Somatic cough syndrome (previously referred to as psychogenic cough) and tic cough (previously referred to as habit cough) in adults and children: CHEST guideline and Expert Panel report; 2015.
31.
American College of Emergency Physicians. Clinical policy: use of intravenous tissue plasminogen activator for the management of acute ischemic stroke in the emergency department; 2015.
32.
American Dental Association. The use of prophylactic antibiotics prior to dental procedures in patients with prosthetic joints: evidence-based clinical practice guideline for dental practitioners—a report of the American Dental Association Council on Scientific Affairs; 2015.
33.
American Geriatrics Society. American Geriatrics Society 2015 updated Beers Criteria for potentially inappropriate medication use in older adults; 2015.
34.
Stav W. Occupational therapy Occupational therapy practice guidelines for driving and community mobility for older adults. Bethesda (MD)Ñ American Occupational Therapy Association (AOTA); 2015.
35.
American Psychiatric Association. Practice guidelines for the psychiatric evaluation of adults, third edition; 2015.
36.
American Society of Anesthesiologists. Practice guidelines for perioperative blood management: an updated report by the American Society of Anesthesiologists Task Force on Perioperative Blood Management; 2015.
37.
American Society of Anesthesiologists. Practice advisory on anesthetic care for magnetic resonance imaging: an updated report by the American Society of Anesthesiologists Task Force on Anesthetic Care for Magnetic Resonance Imaging; 2015.
38.
American Society of Clinical Oncology. Recommendations for the use of WBC growth factors: American Society of Clinical Oncology clinical practice guideline update; 2015.
39.
American Society of Clinical Oncology. Systemic therapy for stage IV non-small-cell lung cancer: American Society of Clinical Oncology clinical practice guideline update; 2015.
40.
American Society of Clinical Oncology. Use of biomarkers to guide decisions on systemic therapy for women with metastatic breast cancer: American Society of Clinical Oncology clinical practice guideline; 2015.
41.
Raghu G, Rochwerg B, Zhang Y, Garcia CA, AzumaA BJ, et al. An official ATS/ERS/JRS/ALAT clinical practice guideline: treatment of idiopathic pulmonary fibrosis. An update of the 2011 clinical practice guideline. Am J Respir Crit Care Med. 2015;192(2):e3–19.CrossRefPubMed
42.
American Urological Association. Castration-resistant prostate cancer: AUA guideline; 2015.
43.
Atkins DL, Berger S, Duff JP, Gonzales JC, Hunt EA, Joyner BL, et al. Part 11: pediatric basic life support and cardiopulmonary resuscitation quality: 2015 American Heart Association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care (reprint). Circulation. 2015;132(18 Suppl 2):S519–25.CrossRefPubMed
44.
Black A, Guilbert E, Co Authors, Costescu D, Dunn S, Fisher W, et al. Canadian contraception consensus (part 2 of 4). J Obstet Gyneacol Can 2015;37(11):1033-1039.
45.
Black A, Guilbert E, Co Authors, Costescu D, Dunn S, Fisher W, et al. Canadian contraception consensus (part 1 of 4). J Obstet Gyneacol Can. 2015;37(10):936-942.
46.
British HIV Association. British HIV Association guidelines for the treatment of HIV-1-positive adults with antiretroviral therapy 2015; 2015.
47.
Canadian Task Force on Preventive Health Care. Recommendations for prevention of weight gain and use of behavioural and pharmacological interventions to manage overweight and obesity in adults in primary care; 2015.
48.
CancerControl Alberta. Systemic therapy for well and moderately differentiated unresectable pancreatic neuroendocrine tumours; 2015.
49.
CancerControl Alberta. Systemic therapy for unresectable stage III or metastatic cutaneous melanoma; 2015.
50.
CancerControl Alberta. Merkel cell carcinoma; 2015.
51.
Belgian Health Care Knowledge Centre. What are the recommended clinical assessment and screening tests during pregnancy?; 2015.
52.
Department of Defense/Department of Veterans Affairs. VA/DoD clinical practice guideline for the management of substance use disorders; 2015.
53.
Eastern Association for the Surgery of Trauma. Evaluation and management of blunt traumatic aortic injury: a practice management guideline from the Eastern Association for the Surgery of Trauma; 2015.
54.
National Institute for Health and Care Excellence. Tuberculosis (CG117); 2015.
55.
National Institute for Health and Care Excellence. Diabetes in pregnancy (CG63); 2015.
56.
National Institute for Health and Care Excellence. Diabetes in children and young people (CG); 2015.
57.
National Institute for Health and Care Excellence. Type 1 diabetes (CG15); 2015.
58.
National Institute for Health and Care Excellence. Referral for suspected cancer (CG27); 2015.
59.
National Institute for Health and Care Excellence. Coeliac disease (CG86); 2015.
60.
National Institute for Health and Care Excellence. Type 2 diabetes: the management of type 2 diabetes (update) (CG66); 2015.
61.
National Institute for Health and Care Excellence. Type 2 diabetes—footcare (CG10); 2015.
62.
National Institute for Health and Care Excellence. Violence (CG25); 2015.
63.
Siu AL, U.S. Preventive Services Task Force. Screening for iron deficiency anemia and iron supplementation in pregnant women to improve maternal health and birth outcomes: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2015;163(7):529–36.CrossRefPubMed
64.
Siu AL, U.S. Preventive Services Task Force. Screening for iron deficiency anemia in young children: USPSTF recommendation statement. Pediatrics. 2015;136(4):746–52.CrossRefPubMed
65.
Siu AL, U.S. Preventive Services Task Force. Screening for abnormal blood glucose and type 2 diabetes mellitus: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2015;163(11):861–8.CrossRefPubMed
66.
Siu AL, U.S. Preventive Services Task Force. Screening for high blood pressure in adults: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2015;163(10):778–86.CrossRefPubMed
67.
Siu AL, U.S. Preventive Services Task Force. Behavioral and pharmacotherapy interventions for tobacco smoking cessation in adults, including pregnant women: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2015;163(8):622–34.CrossRefPubMed
68.
Malaysia Health Technology Assessment Section (MaHTAS). Management of cervical cancer (second edition); 2015.
69.
Malaysia Health Technology Assessment Section (MaHTAS). Management of type 2 diabetes mellitus (5th edition); 2015.
70.
Malaysia Health Technology Assessment Section (MaHTAS). Management of dengue infection in adults (third edition); 2015.
71.
Miyazaki M, Yoshitomi H, Miyakawa S, Uesaka K, Unno M, Endo I, et al. Clinical practice guidelines for the management of biliary tract cancers 2015: the 2nd English edition. J Hepatobiliary Pancreat Sci. 2015;22(4):249–73.CrossRefPubMed
72.
National Clinical Guideline Centre. Anaemia management in people with chronic kidney disease; 2015.
73.
National Collaborating Centre for Nursing and Supportive Care. Irritable bowel syndrome in adults: diagnosis and management of irritable bowel syndrome in primary care; 2015.
74.
National Collaborating Centre for Women. Diabetes (type 1 and type 2) in children and young people: diagnosis and management; 2015.
75.
National Institute for Health and Care Excellence. Prophylaxis against infective endocarditis: antimicrobial prophylaxis against infective endocarditis in adults and children undergoing interventional procedures; 2015.
76.
National Kidney Foundation. KDOQI clinical practice guideline for hemodialysis adequacy: 2015 update; 2015.
77.
Scottisch Intercollegiate Guidelines Network (SIGN). Diagnosis and management of epilepsy in adults (SIGN CPG 143); 2015.
78.
Lyman GH, Bohlke K, Khorana AA, Kuderer NM, Lee AY, Arcelus JI, et al. Venous thromboembolism prophylaxis and treatment in patients with cancer: American Society of Clinical Oncology practice guideline update 2014. J Clin Oncol. 2015;33(6):654–6.CrossRefPubMedPubMedCentral
79.
Hesketh PJ, Bohlke K, Lyman GH, Basch E, Chesney M, Clark-Snow RA, et al. Antiemetics: American Society of Clinical Oncology focused guideline update. J Clin Oncol. 2016;34(4):381–6.CrossRefPubMed
80.
Hwang JP, Somerfield MR, De A-J, Cryer DR, Feld JJ, Kramer BS, et al. Hepatitis B virus screening for patients with cancer before therapy: American Society of Clinical Oncology provisional clinical opinion update. J Clin Oncol. 2015;33(19):2212–20.CrossRefPubMedPubMedCentral
81.
Allegra CJ, Rumble RB, Hamilton SR, Mangu PB, Roach N, Hantel A, et al. Extended RAS gene mutation testing in metastatic colorectal carcinoma to predict response to anti-epidermal growth factor receptor monoclonal antibody therapy: American Society of Clinical Oncology provisional clinical opinion update 2015. J Clin Oncol. 2016;34(2):179–85.CrossRefPubMed
82.
Park IU, Introcaso C, Dunne EF. Human papillomavirus and genital warts: a review of the evidence for the 2015 Centers for Disease Control and Prevention sexually transmitted diseases treatment guidelines. Clin Infect Dis. 2015;61(Suppl 8):S849–55.CrossRefPubMed
83.
Prica A, Baldassarre F, Hicks LK, Imrie K, Kouroukis TC, Cheung M, et al. Rituximab in lymphoma and chronic lymphocytic leukemia: a clinical practice guideline, version 3; 2015.
84.
Registered Nurses’ Association of Ontario. Person- and family-centred care; 2015.
85.
Shea B, Boers M, Grimshaw JM, Hamel C, Bouter LM. Does updating improve the methodological and reporting quality of systematic reviews? BMC Med Res Methodol. 2006;6:27.CrossRefPubMedPubMedCentral
86.
Hasenfeld R, Shekelle PG. Is the methodological quality of guidelines declining in the US? Comparison of the quality of US Agency for Health Care Policy and Research (AHCPR) guidelines with those published subsequently. Qual Saf Health Care. 2003;12(6):428–34.CrossRefPubMedPubMedCentral
87.
Brandt L, Vandvik PO, Alonso-Coello P, Akl EA, Thornton J, Rigua D, et al. Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial. BMJ Open. 2017;7(2):e011569.CrossRefPubMedPubMedCentral
88.
Kastner M, Bhattacharyya O, Hayden L, Makarski J, Estey E, Durocher L, et al. Guideline uptake is influenced by six implementability domains for creating and communicating guidelines: a realist review, 498. J Clin Epidemiol. 68(5):–509.
89.
Newberry SJ, Shekelle PG, Vaiana M, Motala A. Reporting the findings of updated systematic reviews of comparative effectiveness: how do users want to view new information? Rockville: Agency for Healthcare Research and Quality; 2013.
90.
Schunemann HJ, Al-Ansary LA, Forland F, Kersten S, Komulainen J, Kopp IB, et al. Guidelines international network: principles for disclosure of interests and management of conflicts in guidelines. Ann Intern Med. 2015;163(7):548–53.CrossRefPubMed
91.
Armstrong JJ, Goldfarb AM, Instrum RS, MacDermid JC. Improvement evident but still necessary in clinical practice guideline quality: a systematic review. J Clin Epidemiol. 2017;81:13–21.CrossRefPubMed
92.
Alonso-Coello P, Irfan A, Sola I, Gich I, Delgado-Noguera M, Rigau D, et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010;19(6):e58.PubMed
93.
Murad MH. Clinical practice guidelines: a primer on development and dissemination. Mayo Clin Proc. 2017;92(3):423–33.CrossRefPubMed
94.
Alonso-Coello P, Martinez Garcia L, Carrasco JM, Solà I, Qureshi S, Burgers JS, et al. The updating of clinical practice guidelines: insights from an international survey. Implement Sci. 2011;6:107.CrossRefPubMedPubMedCentral
95.
Becker M, Neugebauer EA, Eikermann M. Partial updating of clinical practice guidelines often makes more sense than full updating: a systematic review on methods and the development of an updating procedure. J Clin Epidemiol. 2014;67(1):33–45.CrossRefPubMed
96.
National Institute for Health and Care Excellence. Developing NICE guidelines: the manual [internet]. London: National Institute for Health and Care Excellence (NICE; 2015.
97.
Chan WV, Pearson TA, Bennett GC, Cushman WC, Gaziano TA, Gorman PN, et al. ACC/AHA special report: clinical practice guideline implementation strategies: a summary of systematic reviews by the NHLBI implementation science work group: a report of the American College of Cardiology/American Heart Association Task Force on clinical practice guidelines. J Am Coll Cardiol. 2017;69(8):1076–92.CrossRefPubMed
98.
Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE evidence to decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: clinical practice guidelines. BMJ. 2016;i2089:353.
99.
Martínez García L, Pardo-Hernandez H, Sanabria A, Alonso-Coello P, Pregnancy Clinical Guideline Updating Working Group. Continuous surveillance of a pregnancy clinical guideline: an early experience. Systematic Reviews. 2017;6:143.CrossRefPubMedPubMedCentral
Metadata
Title
Updated clinical guidelines experience major reporting limitations
Authors
Robin W.M. Vernooij
Laura Martínez García
Ivan Dario Florez
Laura Hildago Armas
Michiel H.F. Poorthuis
Melissa Brouwers
Pablo Alonso-Coello
Publication date
01-12-2017
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2017
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-017-0651-3

Other articles of this Issue 1/2017

Implementation Science 1/2017 Go to the issue

Research

HIV policy implementation in two health and demographic surveillance sites in Uganda: findings from a national policy review, health facility surveys and key informant interviews

Research

The effectiveness of a de-implementation strategy to reduce low-value blood management techniques in primary hip and knee arthroplasty: a pragmatic cluster-randomized controlled trial

Study protocol

Improving and sustaining delivery of CPT for PTSD in mental health systems: a cluster randomized trial

Systematic review

Specification of implementation interventions to address the cascade of HIV care and treatment in resource-limited settings: a systematic review

Short report

Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping

Study protocol

Enhancing evidence-based diabetes and chronic disease control among local health departments: a multi-phase dissemination study with a stepped-wedge cluster randomized trial component