Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2005

Open Access 01-12-2005 | Research article

The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation

Authors: Richard N Shiffman, Jane Dixon, Cynthia Brandt, Abdelwaheb Essaihi, Allen Hsiao, George Michel, Ryan O'Connell

Published in: BMC Medical Informatics and Decision Making | Issue 1/2005

Login to get access

Abstract

Background

Clinical practice guidelines are not uniformly successful in influencing clinicians' behaviour toward best practices. Implementability refers to a set of characteristics that predict ease of (and obstacles to) guideline implementation. Our objective is to develop and validate a tool for appraisal of implementability of clinical guidelines.

Methods

Indicators of implementability were identified from the literature and used to create items and dimensions of the GuideLine Implementability Appraisal (GLIA). GLIA consists of 31 items, arranged into 10 dimensions. Questions from 9 of the 10 dimensions are applied individually to each recommendation of the guideline. Decidability and Executability are critical dimensions. Other dimensions are Global, Presentation and Formatting, Measurable Outcomes, Apparent Validity, Flexibility, Effect on Process of Care, Novelty/Innovation, and Computability. We conducted a series of validation activities, including validation of the construct of implementability, expert review of content for clarity, relevance, and comprehensiveness, and assessment of construct validity of the instrument. Finally, GLIA was applied to a draft guideline under development by national professional societies.

Results

Evidence of content validity and preliminary support for construct validity were obtained. The GLIA proved to be useful in identifying barriers to implementation in the draft guideline and the guideline was revised accordingly.

Conclusion

GLIA may be useful to guideline developers who can apply the results to remedy defects in their guidelines. Likewise, guideline implementers may use GLIA to select implementable recommendations and to devise implementation strategies that address identified barriers. By aiding the design and operationalization of highly implementable guidelines, our goal is that application of GLIA may help to improve health outcomes, but further evaluation will be required to support this potential benefit.
Appendix
Available only for authorised users
Literature
1.
go back to reference Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8 (6): 1-72. ReviewCrossRef Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8 (6): 1-72. ReviewCrossRef
2.
go back to reference Larson E: Status of practice guidelines in the United States: CDC guidelines as an example. [Review] [64 refs]. Preventive Medicine. 2003, 36 (5): 519-524. 10.1016/S0091-7435(03)00014-8.CrossRefPubMed Larson E: Status of practice guidelines in the United States: CDC guidelines as an example. [Review] [64 refs]. Preventive Medicine. 2003, 36 (5): 519-524. 10.1016/S0091-7435(03)00014-8.CrossRefPubMed
3.
go back to reference Burgers JS, Grol R, Klazinga NS, Makela M, Zaat J, AGREE Collaboration: Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003, 15 (1): 31-45. 10.1093/intqhc/15.1.31.CrossRefPubMed Burgers JS, Grol R, Klazinga NS, Makela M, Zaat J, AGREE Collaboration: Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003, 15 (1): 31-45. 10.1093/intqhc/15.1.31.CrossRefPubMed
4.
go back to reference Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D: Effect of computerised evidence-based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002, 325: 941-948. 10.1136/bmj.325.7370.941.CrossRefPubMedPubMedCentral Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D: Effect of computerised evidence-based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002, 325: 941-948. 10.1136/bmj.325.7370.941.CrossRefPubMedPubMedCentral
5.
go back to reference Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD: Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003, 18 (12): 967-976. 10.1111/j.1525-1497.2003.30635.x.CrossRefPubMedPubMedCentral Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD: Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003, 18 (12): 967-976. 10.1111/j.1525-1497.2003.30635.x.CrossRefPubMedPubMedCentral
6.
go back to reference Switzer GE, Halm EA, Chang CC, Mittman BS, Walsh MB, Fine MJ: Physician awareness and self-reported use of local and national guidelines for community-acquired pneumonia. J Gen Intern Med. 2003, 18 (10): 816-823. 10.1046/j.1525-1497.2003.20535.x.CrossRefPubMedPubMedCentral Switzer GE, Halm EA, Chang CC, Mittman BS, Walsh MB, Fine MJ: Physician awareness and self-reported use of local and national guidelines for community-acquired pneumonia. J Gen Intern Med. 2003, 18 (10): 816-823. 10.1046/j.1525-1497.2003.20535.x.CrossRefPubMedPubMedCentral
7.
go back to reference Tierney WM, Overhage JM, Takesue BY, Harris LE, Murray MD, Vargo DL, McDonald CJ: Computerizing guidelines to improve care and patient outcomes: the example of heart failure. J Am Med Inform Assoc. 1995, 2: 316-322.CrossRefPubMedPubMedCentral Tierney WM, Overhage JM, Takesue BY, Harris LE, Murray MD, Vargo DL, McDonald CJ: Computerizing guidelines to improve care and patient outcomes: the example of heart failure. J Am Med Inform Assoc. 1995, 2: 316-322.CrossRefPubMedPubMedCentral
8.
go back to reference Katz DA: Barriers between guidelines and improved patient care: an analysis of AHCPR's unstable angina clinical practice guideline. Health Serv Res. 1999, 34 (1): 377-389.PubMedPubMedCentral Katz DA: Barriers between guidelines and improved patient care: an analysis of AHCPR's unstable angina clinical practice guideline. Health Serv Res. 1999, 34 (1): 377-389.PubMedPubMedCentral
9.
go back to reference Institute of Medicine: Guidelines for clinical practice: from development to use. Edited by: Field MJ, Lohr KN. 1992, Washington, DC , National Academy Press Institute of Medicine: Guidelines for clinical practice: from development to use. Edited by: Field MJ, Lohr KN. 1992, Washington, DC , National Academy Press
10.
go back to reference Thorsen T, Mäkelä M: Changing professional practice: theory and practice of clinical guidelines implementation. 1999, Copenhagen , Danish Institute for Health Services Research and Development, 216- Thorsen T, Mäkelä M: Changing professional practice: theory and practice of clinical guidelines implementation. 1999, Copenhagen , Danish Institute for Health Services Research and Development, 216-
11.
go back to reference Solberg LI, Brekke ML, Fowles J, Jacobsen DN, Kottke TE, Mosser G, O'Connor PJ, Ohnsorg KA, Rolnick SJ: Lessons from experienced guideline implementers:attend to many factors and use multiple strategies. Jt Comm J Qual Improv. 2000, 26 (4): 171-188.PubMed Solberg LI, Brekke ML, Fowles J, Jacobsen DN, Kottke TE, Mosser G, O'Connor PJ, Ohnsorg KA, Rolnick SJ: Lessons from experienced guideline implementers:attend to many factors and use multiple strategies. Jt Comm J Qual Improv. 2000, 26 (4): 171-188.PubMed
12.
go back to reference Grilli R, Lomas J: Evaluating the message: the relationship betweenm the compliance rate and the subject of a practice guideline. Medical Care. 1994, 32: 202-213.CrossRefPubMed Grilli R, Lomas J: Evaluating the message: the relationship betweenm the compliance rate and the subject of a practice guideline. Medical Care. 1994, 32: 202-213.CrossRefPubMed
13.
go back to reference Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H: Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998, 317: 858-861.CrossRefPubMedPubMedCentral Grol R, Dalhuijsen J, Thomas S, Veld C, Rutten G, Mokkink H: Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ. 1998, 317: 858-861.CrossRefPubMedPubMedCentral
14.
go back to reference Cluzeau FA, Littlejohns P, Grimshaw JM, Feder G, Moran SE: Development and application of a generic methodology to assess the quality of clinical guidelines. Int J Qual Health Care. 1999, 11 (1): 21-28. 10.1093/intqhc/11.1.21.CrossRefPubMed Cluzeau FA, Littlejohns P, Grimshaw JM, Feder G, Moran SE: Development and application of a generic methodology to assess the quality of clinical guidelines. Int J Qual Health Care. 1999, 11 (1): 21-28. 10.1093/intqhc/11.1.21.CrossRefPubMed
15.
go back to reference AGREE Collaboration: Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003, 12: 18-23. 10.1136/qhc.12.1.18.CrossRef AGREE Collaboration: Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003, 12: 18-23. 10.1136/qhc.12.1.18.CrossRef
16.
go back to reference Shaneyfelt TM, Mayo-Smith MF, Rothwangl J: Are guidelines following guidelines? the methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999, 281: 1900-1905. 10.1001/jama.281.20.1900.CrossRefPubMed Shaneyfelt TM, Mayo-Smith MF, Rothwangl J: Are guidelines following guidelines? the methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999, 281: 1900-1905. 10.1001/jama.281.20.1900.CrossRefPubMed
17.
go back to reference Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM: Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med. 2003, 139: 493-498.CrossRefPubMed Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM: Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization. Ann Intern Med. 2003, 139: 493-498.CrossRefPubMed
19.
go back to reference Anonymous: Practice parameter: the diagnosis, treatment, and evaluation of the initial urinary tract infection in febrile infants and young children. American Academy of Pediatrics. Committee on Quality Improvement. Subcommittee on Urinary Tract Infection [published errata appear in Pediatrics 1999 May;103(5 Pt 1):1052, 1999 Jul;104(1 Pt 1):118 and 2000 Jan;105(1 Pt 1):141] [see comments]. Pediatrics. 1999, 103 (4 Pt 1): 843-852. Anonymous: Practice parameter: the diagnosis, treatment, and evaluation of the initial urinary tract infection in febrile infants and young children. American Academy of Pediatrics. Committee on Quality Improvement. Subcommittee on Urinary Tract Infection [published errata appear in Pediatrics 1999 May;103(5 Pt 1):1052, 1999 Jul;104(1 Pt 1):118 and 2000 Jan;105(1 Pt 1):141] [see comments]. Pediatrics. 1999, 103 (4 Pt 1): 843-852.
20.
go back to reference Linzer M, Yang EH, Estes M, Wang P, Vorperian VR, Kapoor WN: Diagnosing syncope: Part 2. unexplained syncope. Ann Intern Med. 1997, 127: 76-86.CrossRefPubMed Linzer M, Yang EH, Estes M, Wang P, Vorperian VR, Kapoor WN: Diagnosing syncope: Part 2. unexplained syncope. Ann Intern Med. 1997, 127: 76-86.CrossRefPubMed
21.
go back to reference Lynn MR: Determination and quantification of content validity. Nursing Research. 1986, 35: 382-385.CrossRefPubMed Lynn MR: Determination and quantification of content validity. Nursing Research. 1986, 35: 382-385.CrossRefPubMed
22.
go back to reference Anonymous: Otitis media with effusion. Pediatrics. 2004, 113 (5): 1412-1429. 10.1542/peds.113.5.1412.CrossRef Anonymous: Otitis media with effusion. Pediatrics. 2004, 113 (5): 1412-1429. 10.1542/peds.113.5.1412.CrossRef
24.
go back to reference Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M: Are nonspecific practice guidelines potentially harmful? a randomized trial of the effect of nonspecific versus specific guidelines on physician decision making. Health Services research. 2000, 34: 1429-1448.PubMedPubMedCentral Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M: Are nonspecific practice guidelines potentially harmful? a randomized trial of the effect of nonspecific versus specific guidelines on physician decision making. Health Services research. 2000, 34: 1429-1448.PubMedPubMedCentral
25.
go back to reference Greco PJ, Eisenberg JM: Changing physicians' practices. N Engl J Med. 1993, 329: 1271-1274. 10.1056/NEJM199310213291714.CrossRefPubMed Greco PJ, Eisenberg JM: Changing physicians' practices. N Engl J Med. 1993, 329: 1271-1274. 10.1056/NEJM199310213291714.CrossRefPubMed
26.
go back to reference Hunt DL, Haynes RB, Hanna SE, Smith K: Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998, 280: 1339-1346. 10.1001/jama.280.15.1339.CrossRefPubMed Hunt DL, Haynes RB, Hanna SE, Smith K: Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998, 280: 1339-1346. 10.1001/jama.280.15.1339.CrossRefPubMed
27.
go back to reference Shiffman RN, Liaw Y, Brandt CA, Corb GJ: Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc. 1999, 6: 104-114.CrossRefPubMedPubMedCentral Shiffman RN, Liaw Y, Brandt CA, Corb GJ: Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc. 1999, 6: 104-114.CrossRefPubMedPubMedCentral
28.
go back to reference Kaplan B: Evaluating informatics applications--clinical decision support systems literature review. Int J Med Inform. 2001, 64 (1): 15-37. 10.1016/S1386-5056(01)00183-6. ReviewCrossRefPubMed Kaplan B: Evaluating informatics applications--clinical decision support systems literature review. Int J Med Inform. 2001, 64 (1): 15-37. 10.1016/S1386-5056(01)00183-6. ReviewCrossRefPubMed
29.
go back to reference Graham ID, Calder LA, Hebert PC, Carter AO, Tetroe JM: A comparison of clinical practice guideline appraisal instruments. Int J Tech Assess Health Care. 2000, 16: 1024-1038. 10.1017/S0266462300103095.CrossRef Graham ID, Calder LA, Hebert PC, Carter AO, Tetroe JM: A comparison of clinical practice guideline appraisal instruments. Int J Tech Assess Health Care. 2000, 16: 1024-1038. 10.1017/S0266462300103095.CrossRef
30.
go back to reference Ollenschlager G, Marshall C, Qureshi S, Rosenbrand K, Slutsky J, Burgers J, Makela M: Board of Trustees 2002, Guidelines International Network (G-I-N) Improving the quality of health care: using international collaboration to inform guideline programmes - by founding the Guidelines International Network (G-I-N). Qual Safe Health Care. 2004, 13: 455-460. 10.1136/qshc.2003.009761.CrossRef Ollenschlager G, Marshall C, Qureshi S, Rosenbrand K, Slutsky J, Burgers J, Makela M: Board of Trustees 2002, Guidelines International Network (G-I-N) Improving the quality of health care: using international collaboration to inform guideline programmes - by founding the Guidelines International Network (G-I-N). Qual Safe Health Care. 2004, 13: 455-460. 10.1136/qshc.2003.009761.CrossRef
31.
go back to reference Shiffman RN: Representation of clinical practice guidelines in conventional and augmented decision tables. J Am Med Inform Assoc. 1997, 4 (5): 382-393.CrossRefPubMedPubMedCentral Shiffman RN: Representation of clinical practice guidelines in conventional and augmented decision tables. J Am Med Inform Assoc. 1997, 4 (5): 382-393.CrossRefPubMedPubMedCentral
Metadata
Title
The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation
Authors
Richard N Shiffman
Jane Dixon
Cynthia Brandt
Abdelwaheb Essaihi
Allen Hsiao
George Michel
Ryan O'Connell
Publication date
01-12-2005
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2005
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/1472-6947-5-23

Other articles of this Issue 1/2005

BMC Medical Informatics and Decision Making 1/2005 Go to the issue