Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2015

Open Access 01-12-2015 | Research

The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels

Authors: Ignacio Neumann, Romina Brignardello-Petersen, Wojtek Wiercioch, Alonso Carrasco-Labra, Carlos Cuello, Elie Akl, Reem A. Mustafa, Waleed Al-Hazzani, Itziar Etxeandia-Ikobaltzeta, Maria Ximena Rojas, Maicon Falavigna, Nancy Santesso, Jan Brozek, Alfonso Iorio, Pablo Alonso-Coello, Holger J. Schünemann

Published in: Implementation Science | Issue 1/2015

Login to get access

Abstract

Background

Judgments underlying guideline recommendations are seldom recorded and presented in a systematic fashion. The GRADE Evidence-to-Decision Framework (EtD) offers a transparent way to record and report guideline developers’ judgments. In this paper, we report the experiences with the EtD frameworks in 15 real guideline panels.

Methods

Following the guideline panel meetings, we asked methodologists participating in the panel to provide feedback regarding the EtD framework. They were instructed to consider their own experience and the feedback collected from the rest of the panel. Two investigators independently summarized the responses and jointly interpreted the data using pre-specified domains as coding system. We asked methodologists to review the results and provide further input to improve the structure of the EtDs iteratively.

Results

The EtD framework was well received, and the comments were generally positive. Methodologists felt that in a real guideline panel, the EtD framework helps structuring a complex process through relatively simple steps in an explicit and transparent way. However, some sections (e.g., “values and preferences” and “balance between benefits and harms”) required further development and clarification that were considered in the current version of the EtD framework.

Conclusions

The use of an EtD framework in guideline development offers a structured and explicit way to record and report the judgments and discussion of guideline panels during the formulation of recommendations. In addition, it facilitates the formulation of recommendations, assessment of their strength, and identifying gaps in research.
Appendix
Available only for authorised users
Literature
1.
Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527–30.CrossRefPubMedPubMedCentral
2.
Eccles MP, Grimshaw JM, Shekelle P, Schunemann HJ, Woolf S. Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implemen Sci. 2012;7:60.CrossRef
3.
Shekelle P, Woolf S, Grimshaw JM, Schunemann HJ, Eccles MP. Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implemen Sci. 2012;7:62.CrossRef
4.
Woolf S, Schunemann HJ, Eccles MP, Grimshaw JM, Shekelle P. Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implemen Sci. 2012;7:61.CrossRef
5.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. J Clin Epidemiol. 2010;63(12):1308–11.CrossRefPubMed
6.
Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Editors: Robin Graham, Michelle Mancher, Dianne Miller Wolman, Sheldon Greenfield, and Earl Steinberg. Washington (DC): National Academies Press (US); 2011 (available at: http://​www.​nap.​edu/​openbook.​php?​record_​id=​13058)
7.
Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123–142.CrossRefPubMedPubMedCentral
8.
Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, Guyatt GH, Harbour RT, Haugh MC, Henry D, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490.CrossRefPubMed
9.
Schünemann HJ, Best D, Vist G, Oxman AD. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. CMAJ. 2003;169(7):677–80.PubMedPubMedCentral
10.
Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brozek J, Davoli M, Flottorp S, Harbour R, Hill S, et al. Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013;8:6.CrossRefPubMedPubMedCentral
11.
Schunemann HJ, Mustafa R, Brozek J, Santesso N, Alonso-Coello P, Guyatt G, Scholten R, Langendam M, Leeflang MM, Akl EA, et al. GRADE Guidelines: 16. Development of the GRADE Evidence to Decision (EtD) frameworks for tests in clinical practice and public health. J Clin Epidemiol 2016. doi: 10.​1016/​j.​jclinepi.​2016.​01.​032.
12.
Alonso-Coello P, Schunemann H, Moberg J, Brignardello-Petersen R, Akl E, Davoli M, Treweek S, Mustafa R, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AD. GRADE Evidence to Decision frameworks: 1. Introduction. BMJ in press.
13.
Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl E, Davoli M, Treweek S, Mustafa R, Vandvik, P, Meerpohl, J, Guyatt, GH, Schunemann H. GRADE Evidence to Decision frameworks: 2. Clinical practice guidelines. BMJ in press.
14.
15.
Schunemann HJ, Hill SR, Kakad M, Bellamy R, Uyeki TM, Hayden FG, Yazdanpanah Y, Beigel J, Chotpitayasunondh T, Del Mar C, et al. WHO Rapid Advice Guidelines for pharmacological management of sporadic human infection with avian influenza A (H5N1) virus. Lancet Infect Dis. 2007;7(1):21–31.CrossRefPubMed
16.
Schunemann HJ, Hill SR, Kakad M, Vist GE, Bellamy R, Stockman L, Wisloff TF, Del Mar C, Hayden F, Uyeki TM, et al. Transparent development of the WHO rapid advice guidelines. PLoS Med. 2007;4(5):e119.CrossRefPubMedPubMedCentral
17.
Santesso N, Schunemann H, Blumenthal P, De Vuyst H, Gage J, Garcia F, Jeronimo J, Lu R, Luciani S, Quek SC, et al. World Health Organization Guidelines: use of cryotherapy for cervical intraepithelial neoplasia. Int J Gynaecol Obstet. 2012;118(2):97–102.CrossRefPubMed
18.
19.
Schünemann HJ, Brozek J. GRADEpro Guideline Development Tool. In. Hamilton, Canada: McMaster University; 2015.
20.
Schünemann H, Wiecioch W, Brozek J, Etxeandia-Ikobaltzetaen I, Mustafa R, Manja V, Brignadello-Peterson R, Neumann I, Falavigna M, Al-Hazzani W, et al. GRADE Evidence to Decision Frameworks for adoption, adaptation and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT J Clin Epidemiol 2016, accepted for publication.
21.
WHO handbook for guideline development. Geneva, Switzerland: World Health Organization; 2014.
22.
Alonso-Coello P, Irfan A, Sola I, Gich I, Delgado-Noguera M, Rigau D, Tort S, Bonfill X, Burgers J, Schunemann H. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010;19(6):e58.PubMed
23.
24.
25.
Oxman AD, Schunemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 14. Reporting guidelines. Health Res Policy Syst. 2006;4:26.CrossRefPubMedPubMedCentral
26.
Wilson KC, Irwin RS, File Jr TM, Schunemann HJ, Guyatt GH, Rabe KF. Reporting and publishing guidelines: article 12 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proc Am Thorac Soc. 2012;9(5):293–7.CrossRefPubMed
27.
Neumann I, Santesso N, Akl EA, Rind DM, Vandvik PO, Alonso-Coello P, Agoritsas T, Mustafa RA, Alexander PE, Schunemann H, et al. A guide for health professionals to interpret and use recommendations in guidelines developed with the GRADE approach. J Clin Epidemiol. 2016;72:45–55.CrossRefPubMed
28.
Alexander PE, Brito JP, Neumann I, Gionfriddo MR, Bero L, Djulbegovic B, Stoltzfus R, Montori VM, Norris SL, Schunemann HJ, et al. World Health Organization strong recommendations based on low-quality evidence (study quality) are frequent and often inconsistent with GRADE guidance. J Clin Epidemiol. 2016;72:98–106.CrossRefPubMed
29.
Schunemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision? J Clin Epidemiol. 2016;75:6–15.CrossRefPubMed
Metadata
Title
The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels
Authors
Ignacio Neumann
Romina Brignardello-Petersen
Wojtek Wiercioch
Alonso Carrasco-Labra
Carlos Cuello
Elie Akl
Reem A. Mustafa
Waleed Al-Hazzani
Itziar Etxeandia-Ikobaltzeta
Maria Ximena Rojas
Maicon Falavigna
Nancy Santesso
Jan Brozek
Alfonso Iorio
Pablo Alonso-Coello
Holger J. Schünemann
Publication date
01-12-2015
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2015
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-016-0462-y

Other articles of this Issue 1/2015

Implementation Science 1/2015 Go to the issue

Research

Evaluation of a large healthy lifestyle program: informing program implementation and scale-up in the prevention of obesity

Study protocol

The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services

Systematic review

Integrated knowledge translation (IKT) in health care: a scoping review

Reviewer acknowledgement

Annual acknowledgement of manuscript reviewers

Systematic review

A systematic review of the use of the Consolidated Framework for Implementation Research

Research

Tailored implementation of cardiovascular risk management in general practice: a cluster randomized trial