Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
BMC Health Services Research
Published in: BMC Health Services Research 1/2018

Open Access 01-12-2018 | Research article

Guideline appraisal with AGREE II: online survey of the potential influence of AGREE II items on overall assessment of guideline quality and recommendation for use

Authors: Wiebke Hoffmann-Eßer, Ulrich Siering, Edmund A. M. Neugebauer, Anne Catharina Brockhaus, Natalie McGauran, Michaela Eikermann

Published in: BMC Health Services Research | Issue 1/2018

Login to get access

Abstract

Background

The AGREE II instrument is the most commonly used guideline appraisal tool. It includes 23 appraisal criteria (items) organized within six domains. AGREE II also includes two overall assessments (overall guideline quality, recommendation for use). Our aim was to investigate how strongly the 23 AGREE II items influence the two overall assessments.

Methods

An online survey of authors of publications on guideline appraisals with AGREE II and guideline users from a German scientific network was conducted between 10th February 2015 and 30th March 2015. Participants were asked to rate the influence of the AGREE II items on a Likert scale (0 = no influence to 5 = very strong influence). The frequencies of responses and their dispersion were presented descriptively.

Results

Fifty-eight of the 376 persons contacted (15.4%) participated in the survey and the data of the 51 respondents with prior knowledge of AGREE II were analysed. Items 7–12 of Domain 3 (rigour of development) and both items of Domain 6 (editorial independence) had the strongest influence on the two overall assessments. In addition, Items 15–17 (clarity of presentation) had a strong influence on the recommendation for use. Great variations were shown for the other items. The main limitation of the survey is the low response rate.

Conclusions

In guideline appraisals using AGREE II, items representing rigour of guideline development and editorial independence seem to have the strongest influence on the two overall assessments. In order to ensure a transparent approach to reaching the overall assessments, we suggest the inclusion of a recommendation in the AGREE II user manual on how to consider item and domain scores. For instance, the manual could include an a-priori weighting of those items and domains that should have the strongest influence on the two overall assessments. The relevance of these assessments within AGREE II could thereby be further specified.
Appendix
Available only for authorised users
Literature
1.
Field MJ, Lohr KN, editors. Clinical practice guidelines: directions for a new program. Washington: National Academy Press; 1990.
2.
Graham RM, Mancher M, Miller-Wolman D, Greenfield S, Steinberg E, editors. Clinical practice guidelines we can trust. Washington: National Academies Press; 2011.
3.
Hakkennes S, Dodd K. Guideline implementation in allied health professions: a systematic review of the literature. Qual Saf Health Care. 2008;17(4):296–300.CrossRefPubMed
4.
Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8(6):iii–v. 1-72CrossRefPubMed
5.
Medves J, Godfrey C, Turner C, Paterson M, Harrison M, MacKenzie L, Durando P. Systematic review of practice guideline dissemination and implementation strategies for healthcare teams and team-based practice. Int J Evid Based Healthc. 2010;8(2):79–89.PubMed
6.
Ray-Coquard I, Philip T, Lehmann M, Fervers B, Farsi F, Chauvin F. Impact of a clinical guidelines program for breast and colon cancer in a French cancer center. JAMA. 1997;278(19):1591–5.CrossRefPubMed
7.
Smith TJ, Hillner BE. Ensuring quality cancer care by the use of clinical practice guidelines and critical pathways. J Clin Oncol. 2001;19(11):2886–97.CrossRefPubMed
8.
Ray-Coquard I, Philip T, De Laroche G, Froger X, Suchaud JP, Voloch A, Mathieu-Daude H, Fervers B, Farsi F, Browman GP, et al. A controlled "before-after" study: impact of a clinical guidelines programme and regional cancer network organization on medical practice. Br J Cancer. 2002;86(3):313–21.CrossRefPubMedPubMedCentral
9.
Grimshaw J, Eccles M, Tetroe J. Implementing clinical guidelines: current evidence and future implications. J Contin Educ Heal Prof. 2004;24(Suppl 1):S31–7.CrossRef
10.
Alonso-Coello P, Irfan A, Sola I, Gich I, Delgado-Noguera M, Rigau D, Tort S, Bonfill X, Burgers J. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010;19(6):e58.PubMed
11.
Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000;355(9198):103–6.CrossRefPubMed
12.
Kryworuchko J, Stacey D, Bai N, Graham ID. Twelve years of clinical practice guideline development, dissemination and evaluation in Canada (1994 to 2005). Implement Sci. 2009;4:49.CrossRefPubMedPubMedCentral
13.
Kung J, Miller RR, Mackowiak PA. Failure of clinical practice guidelines to meet institute of medicine standards: two more decades of little, if any, progress. Arch Intern Med. 2012;172(21):1628–33.CrossRefPubMed
14.
Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999;281(20):1900–5.CrossRefPubMed
15.
AGREE Collaboration. The appraisal of guidelines for research & evaluation (AGREE) instrument. London: AGREE Research Trust; 2006.
16.
17.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182(18):E839–42.CrossRefPubMedPubMedCentral
18.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Hanna SE, Makarski J. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ. 2010;182(10):1045–52.CrossRefPubMedPubMedCentral
19.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Hanna SE, Makarski J. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ. 2010;182(10):E472–8.CrossRefPubMedPubMedCentral
20.
Brosseau L, Rahman P, Toupin-April K, Poitras S, King J, De Angelis G, Loew L, Casimiro L, Paterson G, McEwan J. A systematic critical appraisal for non-pharmacological management of osteoarthritis using the appraisal of guidelines research and evaluation II instrument. PLoS One. 2014;9(1):e82986.CrossRefPubMedPubMedCentral
21.
Lee GY, Yamada J, Kyololo OB, Shorkey A, Stevens B. Pediatric clinical practice guidelines for acute procedural pain: a systematic review. Pediatrics. 2014;133(3):500–15.CrossRefPubMed
22.
Polus S, Lerberg P, Vogel J, Watananirun K, Souza JP, Mathai M, Gulmezoglu AM. Appraisal of WHO guidelines in maternal health using the AGREE II assessment tool. PLoS One. 2012;7(8):e38891.CrossRefPubMedPubMedCentral
23.
Sabharwal S, Gauher S, Kyriacou S, Patel V, Holloway I, Athanasiou T. Quality assessment of guidelines on thromboprophylaxis in orthopaedic surgery. Bone Joint J. 2014;96-B(1):19–23.CrossRefPubMed
24.
Sabharwal S, Patel NK, Gauher S, Holloway I, Athanasiou T. High methodologic quality but poor applicability: assessment of the AAOS guidelines using the AGREE II instrument. Clin Orthop. 2014;472(6):1982–8.CrossRefPubMedPubMedCentral
25.
Hoffmann-Esser W, Siering U, Neugebauer EA, Brockhaus AC, Lampert U, Eikermann M. Guideline appraisal with AGREE II: systematic review of the current evidence on how users handle the 2 overall assessments. PLoS One. 2017;12(3):e0174831.CrossRefPubMedPubMedCentral
26.
Resource Toolkit for Guideline Adaptation: Version 1.0 [n.a.]
27.
Brosseau L, Rahman P, Poitras S, Toupin-April K, Paterson G, Smith C, King J, Casimiro L, De Angelis G, Loew L, et al. A systematic critical appraisal of non-pharmacological management of rheumatoid arthritis with appraisal of guidelines for research and evaluation II. PLoS One. 2014;9(5):e95369.CrossRefPubMedPubMedCentral
28.
Bindslev JB, Schroll J, Gotzsche PC, Lundh A. Underreporting of conflicts of interest in clinical practice guidelines: cross sectional study. BMC Med Ethics. 2013;14:19.CrossRefPubMedPubMedCentral
29.
Feuerstein J, Gifford A, Akbari M, Goldmann J, Leffler D, Sheth S, Cheifetz A. Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in gastroenterology practice guidelines. Am J Gastroenterol. 2013;108(11):1686–93.CrossRefPubMed
30.
Norris SL, Holmer HK, Ogden LA, Burda BU. Conflict of interest in clinical practice guideline development: a systematic review. PLoS One. 2011;6(10):e25153.CrossRefPubMedPubMedCentral
31.
Fervers B, Burgers JS, Haugh MC, Brouwers M, Browman G, Cluzeau F, Philip T. Predictors of high quality clinical practice guidelines: examples in oncology. Int J Qual Health Care. 2005;17(2):123–32.CrossRefPubMed
32.
Acuna-Izcaray A, Sanchez-Angarita E, Plaza V, Rodrigo G, Montes de Oca M, Gich I, Bonfill X, Alonso-Coello P. Quality assessment of asthma clinical practice guidelines: a systematic appraisal. Chest. 2013;144(2):390–7.CrossRefPubMed
33.
Arevalo-Rodriguez I, Pedraza OL, Rodriguez A, Sanchez E, Gich I, Sola I, Bonfill X, Alonso-Coello P. Alzheimer's disease dementia guidelines for diagnostic testing: a systematic review. Am J Alzheimers Dis Other Demen. 2013;28(2):111–9.CrossRefPubMed
34.
Bekkering GE, Aertgeerts B, Asueta-Lorente JF, Autrique M, Goossens M, Smets K, Van Bussel JC, Vanderplasschen W, Van Royen P, Hannes K. Practitioner review: evidence-based practice guidelines on alcohol and drug misuse among adolescents; a systematic review. J Child Psychol Psychiatry. 2014;55(1):3–21.CrossRefPubMed
35.
Bragge P, Pattuwage L, Marshall S, Pitt V, Piccenna L, Stergiou-Kita M, Tate RL, Teasell R, Wiseman-Hakes C. Quality of guidelines for cognitive rehabilitation following traumatic brain injury. J Head Trauma Rehabil. 2014;29(4):277–89.CrossRefPubMed
36.
Huang TW, Lai JH, Wu MY, Chen SL, Wu CH, Tam KW. Systematic review of clinical practice guidelines in the diagnosis and management of thyroid nodules and cancer. BMC Med. 2013;11:191.CrossRefPubMedPubMedCentral
37.
Kim SG, Jung HK, Lee HL, Jang JY, Lee H, Kim CG, Shin WG, Shin ES, Lee YC. Guidelines for the diagnosis and treatment of helicobacter pylori infection in Korea, 2013 revised edition. J Gastroenterol Hepatol. 2014;29(7):1371–86.CrossRefPubMed
38.
Langton JM, Pearson SA. eviQ cancer treatments online: how does the web-based protocol system fare in a comprehensive quality assessment? Asia Pac J Clin Oncol. 2011;7(4):357–63.CrossRefPubMed
39.
Larmer PJ, Reay ND, Aubert ER, Kersten P. Systematic review of guidelines for the physical management of osteoarthritis. Arch Phys Med Rehabil. 2014;95(2):375–89.CrossRefPubMed
40.
Lopez-Vargas PA, Tong A, Sureshkumar P, Johnson DW, Craig JC. Prevention, detection and management of early chronic kidney disease: a systematic review of clinical practice guidelines. Nephrology. 2013;18(9):592–604.CrossRefPubMed
41.
Norberg MM, Turner MW, Rooke SE, Langton JM, Gates PJ. An evaluation of web-based clinical practice guidelines for managing problems associated with cannabis use. J Med Internet Res. 2012;14(6):e169.CrossRefPubMedPubMedCentral
42.
Parisi P, Vanacore N, Belcastro V, Carotenuto M, Del Giudice E, Mariani R, Papetti L, Pavone P, Savasta S, Striano P. Clinical guidelines in pediatric headache: evaluation of quality using the AGREE II instrument. J Headache Pain. 2014;15(1):57.CrossRefPubMedPubMedCentral
43.
Piano V, Schalkwijk A, Burgers J, Verhagen S, Kress H, Hekster Y, Lanteri-Minet M, Engels Y, Vissers K. Guidelines for neuropathic pain management in patients with cancer: a European survey and comparison. Pain Practice. 2013;13(5):349–57.CrossRefPubMed
44.
Rios E, Seron P, Lanas F, Bonfill X, Quigley EMM, Alonso-Coello P. Evaluation of the quality of clinical practice guidelines for the management of esophageal or gastric variceal bleeding. Eur J Gastroenterol Hepatol. 2014;26(4):422–31.CrossRefPubMed
45.
Rohde A, Worrall L, Le Dorze G. Systematic review of the quality of clinical guidelines for aphasia in stroke management. J Eval Clin Pract. 2013;19(6):994–1003.CrossRefPubMed
46.
Sanclemente G, Acosta JL, Tamayo ME, Bonfill X, Alonso-Coello P. Clinical practice guidelines for treatment of acne vulgaris: a critical appraisal using the AGREE II instrument. Arch Dermatol Res. 2014;306(3):269–77.CrossRefPubMed
47.
Santos F, Sola I, Rigau D, Arevalo-Rodriguez I, Seron P, Alonso-Coello P, Berard A, Bonfill X. Quality assessment of clinical practice guidelines for the prescription of antidepressant drugs during pregnancy. Curr Clin Pharmacol. 2012;7(1):7–14.CrossRefPubMed
48.
Schildmann EK, Schildmann J, Kiesewetter I. Medication and monitoring in palliative sedation therapy: a systematic review and quality assessment of published guidelines. J Pain Symptom Manag. 2015;49(4):734–46.CrossRef
49.
Schoenmaker NJ, Tromp WF, Van der Lee JH, Offringa M, Craig JC, Groothoff JW. Quality and consistency of clinical practice guidelines for the management of children on chronic dialysis. Nephrology Dialysis Transplantation. 2013;28(12):3052–61.CrossRef
50.
Seron P, Lanas F, Rios E, Bonfill X, Alonso-Coello P. Evaluation of the quality of clinical guidelines for cardiac rehabilitation: a critical review. J Cardiopulm Rehabil Prev. 2014;35(1):1–12.CrossRef
51.
Shen J, Sun M, Zhou B, Yan J. Nonconformity in the clinical practice guidelines for subclinical Cushing's syndrome: which guidelines are trustworthy? Eur J Endocrinol. 2014;171(4):421–31.CrossRefPubMed
52.
Siering U, Hoffmann-Eßer W, Neugebauer EA, Eikermann M: Is there a cut-off for high-quality guidelines? A systematic analysis of current guideline appraisals using the AGREE-II instrument [poster]. G-I-N Conference; 7th - 10th October 2015; Amsterdam.
53.
Wang Y, Luo Q, Li Y, Wang H, Deng S, Wei S, Li X. Quality assessment of clinical practice guidelines on the treatment of hepatocellular carcinoma or metastatic liver cancer. PLoS One. 2014;9(8):e103939.CrossRefPubMedPubMedCentral
54.
Yan J, Min J, Zhou B. Diagnosis of pheochromocytoma: a clinical practice guideline appraisal using AGREE II instrument. J Eval Clin Pract. 2013;19(4):626–32.CrossRefPubMed
55.
56.
Burnett HF, Tanoshima R, Chandranipapongse W, Madadi P, Ito S, Ungar WJ. Testing for thiopurine methyltransferase status for safe and effective thiopurine administration: a systematic review of clinical guidance documents. Pharmacogenomics J. 2014;14(6):493–502.CrossRefPubMed
57.
Haran C, Van Driel M, Mitchell BL, Brodribb WE. Clinical guidelines for postpartum women and infants in primary care: a systematic review. BMC Pregnancy Childbirth. 2014;14:51.CrossRefPubMedPubMedCentral
58.
59.
White PE, Shee AW, Finch CF. Independent appraiser assessment of the quality, methodological rigour and transparency of the development of the 2008 international consensus statement on concussion in sport. Br J Sports Med. 2014;48(2):130–4.CrossRefPubMed
60.
Al-Ansary LA, Tricco AC, Adi Y, Bawazeer G, Perrier L, Al-Ghonaim M, AlYousefi N, Tashkandi M, Straus SE. A systematic review of recent clinical practice guidelines on the diagnosis, assessment and management of hypertension. PLoS One. 2013;8(1):e53744.CrossRefPubMedPubMedCentral
61.
Stacey D, Macartney G, Carley M, Harrison MB, Costars TP. Development and evaluation of evidence-informed clinical nursing protocols for remote assessment, triage and support of cancer treatment-induced symptoms. Nurs Res Pract. 2013;2013:171872.PubMedPubMedCentral
62.
Tian H, Gou Y, Pan Y, Li Q, Wei D, Wang Z, Niu X, Liang W, Zhang Y. Quality appraisal of clinical practice guidelines on glioma. Neurosurg Rev. 2015;38(1):39–47.CrossRefPubMed
63.
Halbesleben JR, Whitman MV. Evaluating survey quality in health services research: a decision framework for assessing nonresponse bias. Health Serv Res. 2013;48(3):913–30.CrossRefPubMed
64.
65.
Sinclair M, O'Toole J, Malawaraarachchi M, Leder K. Comparison of response rates and cost-effectiveness for a community-based survey: postal, internet and telephone modes with generic or personalised recruitment approaches. BMC Med Res Methodol. 2012;12:132.CrossRefPubMedPubMedCentral
66.
Metadata
Title
Guideline appraisal with AGREE II: online survey of the potential influence of AGREE II items on overall assessment of guideline quality and recommendation for use
Authors
Wiebke Hoffmann-Eßer
Ulrich Siering
Edmund A. M. Neugebauer
Anne Catharina Brockhaus
Natalie McGauran
Michaela Eikermann
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2018
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/s12913-018-2954-8

Other articles of this Issue 1/2018

BMC Health Services Research 1/2018 Go to the issue

Research article

Being treated in higher volume hospitals leads to longer progression-free survival for epithelial ovarian carcinoma patients in the Rhone-Alpes region of France

Research article

Implementation of the Chronic Disease Care System and its association with health care costs and continuity of care in Korean adults with type 2 diabetes mellitus

Research article

Quality of care for children with acute malnutrition at health center level in Uganda: a cross sectional study in West Nile region during the refugee crisis

Research article

Epidemiology of ambulance utilized patients in Addis Ababa, Ethiopia

Research article

Analysis of health service utilization of migrants in Beijing using Anderson health service utilization model

Research article

Assessment of patient safety culture: a nationwide survey of community pharmacists in Kuwait