Skip to main content
Top
Published in: Implementation Science 1/2014

Open Access 01-12-2014 | Research

A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools

Authors: Anna R Gagliardi, Melissa C Brouwers, Onil K Bhattacharyya

Published in: Implementation Science | Issue 1/2014

Login to get access

Abstract

Background

Guidelines are the foundation for healthcare planning, delivery and quality improvement but are not consistently implemented. Few guidelines are accompanied by guideline implementation tools (GItools). Users have requested GItools, and developers have requested guidance on how to develop GItools. First it is necessary to characterize GItools. The purpose of this research was to generate a framework of desirable features of GItools.

Methods

Items representing desirable GItool features were generated by a cross-sectional survey of the international guideline community. Items were confirmed by 31 guideline developers, implementers and researchers in a two-round Delphi survey administered on the Internet. The resulting GItool framework was applied with a sample of GItools accompanying guidelines identified in the National Guideline Clearinghouse.

Results

The cross-sectional survey was completed by 96 respondents from Australia, Canada, the United Kingdom, the United States, The Netherlands, and various other countries. Seven of nine items were rated by the majority as desirable. A total of 31 panelists from 10 countries including Australia, Canada, Germany, New Zealand, Peru, Saudi Arabia, Spain, the United Kingdom, and the United States took part in a two-round Delphi survey. Ten items achieved consensus as desirable GItool features in round #1, and two additional items in round #2. A total of 13 GItools for Resource Planning, Implementation and Evaluation were identified among 149 guidelines on a variety of clinical topics (8.7%). Many GItools did not possess features considered desirable.

Conclusions

Inclusion of higher quality GItools in guidelines is needed to support user adoption of guidelines. The GItool framework can serve as the basis for evaluating and adapting existing GItools, or developing new GItools. Further research is needed to validate the framework, develop and implement instruments by which developers can apply the framework, and specify which guidelines should be accompanied by GItools.
Literature
1.
go back to reference Weisz G, Cambrosio A, Keating P, Knaapen L, Schlich T, Tournay VJ: The emergence of clinical practice guidelines. Milbank Q. 2007, 85 (4): 691-727. 10.1111/j.1468-0009.2007.00505.x.CrossRefPubMedPubMedCentral Weisz G, Cambrosio A, Keating P, Knaapen L, Schlich T, Tournay VJ: The emergence of clinical practice guidelines. Milbank Q. 2007, 85 (4): 691-727. 10.1111/j.1468-0009.2007.00505.x.CrossRefPubMedPubMedCentral
2.
go back to reference Scott I: The evolving science of translating research evidence into clinical practice. Evid Based Med. 2007, 12: 4-7. 10.1136/ebm.12.1.4.CrossRefPubMed Scott I: The evolving science of translating research evidence into clinical practice. Evid Based Med. 2007, 12: 4-7. 10.1136/ebm.12.1.4.CrossRefPubMed
3.
go back to reference Shekelle P, Woolf S, Grimshaw JM, Schunemann H, Eccles MP: Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci. 2012, 7: 62-10.1186/1748-5908-7-62.CrossRefPubMedPubMedCentral Shekelle P, Woolf S, Grimshaw JM, Schunemann H, Eccles MP: Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci. 2012, 7: 62-10.1186/1748-5908-7-62.CrossRefPubMedPubMedCentral
4.
go back to reference Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, Baldeh T, Zhang Y, Raid U, Neumann I, Norris SL, Thornton J, Harbour R, Treweek S, Guyatt G, Alonso-Coello P, Reinap M, Brozek J, Oxman A, Akl EA: Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014, 186 (3): E123-142. 10.1503/cmaj.131237.CrossRefPubMedPubMedCentral Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, Baldeh T, Zhang Y, Raid U, Neumann I, Norris SL, Thornton J, Harbour R, Treweek S, Guyatt G, Alonso-Coello P, Reinap M, Brozek J, Oxman A, Akl EA: Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014, 186 (3): E123-142. 10.1503/cmaj.131237.CrossRefPubMedPubMedCentral
5.
go back to reference Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Littlejohns P, Makarski J, Zitzelsberger L: AGREE II: Advancing guideline development, reporting and evaluation in health care. J Clin Epidemiol. 2010, 63 (12): 1308-1311. 10.1016/j.jclinepi.2010.07.001.CrossRefPubMed Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Littlejohns P, Makarski J, Zitzelsberger L: AGREE II: Advancing guideline development, reporting and evaluation in health care. J Clin Epidemiol. 2010, 63 (12): 1308-1311. 10.1016/j.jclinepi.2010.07.001.CrossRefPubMed
6.
go back to reference Schunemann HJ, Oxman AD, Brozek J, Glasziou P, Bossuyt P, Chang S, Muti P, Jaeschke R, Guyatt GH: GRADE: assessing the quality of evidence for diagnostic recommendations. Evid Based Med. 2008, 13 (6): 162-163. 10.1136/ebm.13.6.162-a.CrossRefPubMed Schunemann HJ, Oxman AD, Brozek J, Glasziou P, Bossuyt P, Chang S, Muti P, Jaeschke R, Guyatt GH: GRADE: assessing the quality of evidence for diagnostic recommendations. Evid Based Med. 2008, 13 (6): 162-163. 10.1136/ebm.13.6.162-a.CrossRefPubMed
7.
go back to reference Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G, O’Connell R: The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005, 5: 23-10.1186/1472-6947-5-23.CrossRefPubMedPubMedCentral Shiffman RN, Dixon J, Brandt C, Essaihi A, Hsiao A, Michel G, O’Connell R: The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak. 2005, 5: 23-10.1186/1472-6947-5-23.CrossRefPubMedPubMedCentral
8.
go back to reference Siering U, Eikermann M, Hausner E, Hoffman-Eber W, Neugebauer EA: Appraisal tools for clinical practice guidelines: A systematic review. PLoS ONE. 2013, 8: e892915-10.1371/journal.pone.0082915.CrossRef Siering U, Eikermann M, Hausner E, Hoffman-Eber W, Neugebauer EA: Appraisal tools for clinical practice guidelines: A systematic review. PLoS ONE. 2013, 8: e892915-10.1371/journal.pone.0082915.CrossRef
9.
go back to reference McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. NEJM. 2003, 348 (26): 2635-2645. 10.1056/NEJMsa022615.CrossRefPubMed McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. NEJM. 2003, 348 (26): 2635-2645. 10.1056/NEJMsa022615.CrossRefPubMed
10.
go back to reference Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, West P, Wright D, Wright J: What’s the evidence that NICE guidance has been implemented? Results from a national evaluation using time series analysis, audit of patients’ notes, and interviews. BMJ. 2004, 329 (7473): 999-10.1136/bmj.329.7473.999.CrossRefPubMedPubMedCentral Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, West P, Wright D, Wright J: What’s the evidence that NICE guidance has been implemented? Results from a national evaluation using time series analysis, audit of patients’ notes, and interviews. BMJ. 2004, 329 (7473): 999-10.1136/bmj.329.7473.999.CrossRefPubMedPubMedCentral
11.
go back to reference Kryworuchko J, Stacey D, Bai N, Graham ID: Twelve years of clinical practice guideline development, dissemination and evaluation in Canada (1994 to 2005). Implement Sci. 2009, 4: 49-10.1186/1748-5908-4-49.CrossRefPubMedPubMedCentral Kryworuchko J, Stacey D, Bai N, Graham ID: Twelve years of clinical practice guideline development, dissemination and evaluation in Canada (1994 to 2005). Implement Sci. 2009, 4: 49-10.1186/1748-5908-4-49.CrossRefPubMedPubMedCentral
12.
go back to reference Burgers JS, Grol R, Klazinga NS, Makela M, Zaat J: Towards evidence based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003, 15 (1): 31-45. 10.1093/intqhc/15.1.31.CrossRefPubMed Burgers JS, Grol R, Klazinga NS, Makela M, Zaat J: Towards evidence based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003, 15 (1): 31-45. 10.1093/intqhc/15.1.31.CrossRefPubMed
13.
go back to reference Gagliardi AR: “More bang for the buck”: exploring optimal approaches for guideline implementation through interviews with international developers. BMC Health Serv Res. 2012, 12: 404-10.1186/1472-6963-12-404.CrossRefPubMedPubMedCentral Gagliardi AR: “More bang for the buck”: exploring optimal approaches for guideline implementation through interviews with international developers. BMC Health Serv Res. 2012, 12: 404-10.1186/1472-6963-12-404.CrossRefPubMedPubMedCentral
14.
go back to reference Mickan S, Burls A, Glasziou P: Patterns of “leakage” in the utilization of clinical guidelines: a systematic review. Postgrad Med J. 2011, 87 (1032): 670-679. 10.1136/pgmj.2010.116012.CrossRefPubMedPubMedCentral Mickan S, Burls A, Glasziou P: Patterns of “leakage” in the utilization of clinical guidelines: a systematic review. Postgrad Med J. 2011, 87 (1032): 670-679. 10.1136/pgmj.2010.116012.CrossRefPubMedPubMedCentral
15.
go back to reference McKillop A, Crisp J, Walsh K: Practice guidelines need to address the “how” and the “what” of implementation. Primary Health Care Res Dev. 2012, 13 (1): 48-59. 10.1017/S1463423611000405.CrossRef McKillop A, Crisp J, Walsh K: Practice guidelines need to address the “how” and the “what” of implementation. Primary Health Care Res Dev. 2012, 13 (1): 48-59. 10.1017/S1463423611000405.CrossRef
16.
go back to reference Pronovost PJ: Enhancing physicians’ use of clinical guidelines. JAMA. 2013, 310 (23): 2501-2. 10.1001/jama.2013.281334.CrossRefPubMed Pronovost PJ: Enhancing physicians’ use of clinical guidelines. JAMA. 2013, 310 (23): 2501-2. 10.1001/jama.2013.281334.CrossRefPubMed
17.
go back to reference Patel VL, Arocha JF, Diermeier M, Greenes RA, Shortliffe EH: Methods of cognitive analysis to support the design and evaluation of biomedical systems. J Biomed Inform. 2001, 34 (1): 52-66. 10.1006/jbin.2001.1002.CrossRefPubMed Patel VL, Arocha JF, Diermeier M, Greenes RA, Shortliffe EH: Methods of cognitive analysis to support the design and evaluation of biomedical systems. J Biomed Inform. 2001, 34 (1): 52-66. 10.1006/jbin.2001.1002.CrossRefPubMed
18.
go back to reference Cochrane LJ, Olson CA, Murray S, Dupuis M, Tooman T, Hayes S: Gaps between knowing and doing: Understanding and assessing the barriers to optimal health care. J Cont Ed Health Prof. 2007, 27 (2): 94-102. 10.1002/chp.106.CrossRef Cochrane LJ, Olson CA, Murray S, Dupuis M, Tooman T, Hayes S: Gaps between knowing and doing: Understanding and assessing the barriers to optimal health care. J Cont Ed Health Prof. 2007, 27 (2): 94-102. 10.1002/chp.106.CrossRef
19.
go back to reference Dobbins M, Hanna SE, Ciliska D, Manske S, Cameron R, Mercer SL, O’Mara L, DeCorby K, Robeson P: A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Implement Sci. 2009, 4: 61-10.1186/1748-5908-4-61.CrossRefPubMedPubMedCentral Dobbins M, Hanna SE, Ciliska D, Manske S, Cameron R, Mercer SL, O’Mara L, DeCorby K, Robeson P: A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Implement Sci. 2009, 4: 61-10.1186/1748-5908-4-61.CrossRefPubMedPubMedCentral
20.
go back to reference Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M: Are non-specific guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Health Serv Res. 2000, 34 (7): 1429-1448.PubMedPubMedCentral Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M: Are non-specific guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Health Serv Res. 2000, 34 (7): 1429-1448.PubMedPubMedCentral
21.
go back to reference Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM: How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011, 6: 26-10.1186/1748-5908-6-26.CrossRefPubMedPubMedCentral Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM: How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011, 6: 26-10.1186/1748-5908-6-26.CrossRefPubMedPubMedCentral
22.
go back to reference Gagliardi AR, Brouwers MC: Integrating guideline development and implementation: Analysis of guideline development manual instructions for generating implementation advice. Implement Sci. 2012, 7: 67-10.1186/1748-5908-7-67.CrossRefPubMedPubMedCentral Gagliardi AR, Brouwers MC: Integrating guideline development and implementation: Analysis of guideline development manual instructions for generating implementation advice. Implement Sci. 2012, 7: 67-10.1186/1748-5908-7-67.CrossRefPubMedPubMedCentral
23.
go back to reference Kelley K, Clark B, Brown V, Sitzia J: Good practice in the conduct and reporting of survey research. Int J Qual Health Care. 2003, 15 (3): 261-266. 10.1093/intqhc/mzg031.CrossRefPubMed Kelley K, Clark B, Brown V, Sitzia J: Good practice in the conduct and reporting of survey research. Int J Qual Health Care. 2003, 15 (3): 261-266. 10.1093/intqhc/mzg031.CrossRefPubMed
25.
go back to reference Streiner DL, Norman GR: Health Measurement Scales. A Practical Guide to Their Development and Use. 2003, Oxford University Press, Oxford (UK) Streiner DL, Norman GR: Health Measurement Scales. A Practical Guide to Their Development and Use. 2003, Oxford University Press, Oxford (UK)
26.
go back to reference Dillman DA, Smyth JD: Design effects in the transition to web-based surveys. Am J Prev Med. 2007, 32 (5 Suppl): S90-S96. 10.1016/j.amepre.2007.03.008.CrossRefPubMed Dillman DA, Smyth JD: Design effects in the transition to web-based surveys. Am J Prev Med. 2007, 32 (5 Suppl): S90-S96. 10.1016/j.amepre.2007.03.008.CrossRefPubMed
27.
go back to reference Fink A, Kosecoff J, Chassin M, Brook RH: Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984, 74 (9): 979-983. 10.2105/AJPH.74.9.979.CrossRefPubMedPubMedCentral Fink A, Kosecoff J, Chassin M, Brook RH: Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984, 74 (9): 979-983. 10.2105/AJPH.74.9.979.CrossRefPubMedPubMedCentral
28.
go back to reference Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C: Using and reporting Delphi method for selecting healthcare quality indicators: a systematic review. PLoS ONE. 2011, 6: e20476-10.1371/journal.pone.0020476.CrossRefPubMedPubMedCentral Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C: Using and reporting Delphi method for selecting healthcare quality indicators: a systematic review. PLoS ONE. 2011, 6: e20476-10.1371/journal.pone.0020476.CrossRefPubMedPubMedCentral
29.
go back to reference Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, Sirett E, Wong G, Cargo M, Herbert CP, Seifer SD, Green LW, Greenhalgh T: Uncovering the benefits of participatory research: Implications of a realist review for health research and practice. Milbank Q. 2012, 90 (2): 311-346. 10.1111/j.1468-0009.2012.00665.x.CrossRefPubMedPubMedCentral Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, Sirett E, Wong G, Cargo M, Herbert CP, Seifer SD, Green LW, Greenhalgh T: Uncovering the benefits of participatory research: Implications of a realist review for health research and practice. Milbank Q. 2012, 90 (2): 311-346. 10.1111/j.1468-0009.2012.00665.x.CrossRefPubMedPubMedCentral
30.
go back to reference Kothari A, Wathen CN: A critical second look at integrated knowledge translation. Health Policy. 2013, 109 (2): 187-191. 10.1016/j.healthpol.2012.11.004.CrossRefPubMed Kothari A, Wathen CN: A critical second look at integrated knowledge translation. Health Policy. 2013, 109 (2): 187-191. 10.1016/j.healthpol.2012.11.004.CrossRefPubMed
Metadata
Title
A framework of the desirable features of guideline implementation tools (GItools): Delphi survey and assessment of GItools
Authors
Anna R Gagliardi
Melissa C Brouwers
Onil K Bhattacharyya
Publication date
01-12-2014
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2014
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-014-0098-8

Other articles of this Issue 1/2014

Implementation Science 1/2014 Go to the issue