Skip to main content
Top
Published in: Implementation Science 1/2018

Open Access 01-12-2018 | Research

Reporting quality of European and Croatian health practice guidelines according to the RIGHT reporting checklist

Authors: Ružica Tokalić, Marin Viđak, Ivan Buljan, Ana Marušić

Published in: Implementation Science | Issue 1/2018

Login to get access

Abstract

Background

Health practice guidelines (HPGs) are important tools for the translation of evidence into practice. Reporting Items for Practice Guidelines in HealThcare (RIGHT) checklist provides guidance on reporting health practice guidelines (HPGs). We assessed the reporting completeness and quality of a set of national (Croatian) and relevant transnational (European) HPGs.

Methods

The national sample included all HPGs published in the official journal of the Croatian Medical Association in 2014–2016. We searched PubMed to identify relevant European guidelines (n = 24). Two independent reviewers assessed the adherence with the items on the RIGHT checklist. Kappa score was used to measure the level of agreement. Frequentist and Bayes statistics Bayes factor (BF10) was used to evaluate the differences between the national and transnational HPGs.

Results

Overall, Croatian and European HPGs adhered to less than 50% of RIGHT checklist items. Croatian HPGs reported a median of 14.0 (95% CI 13.0–15.0) RIGHT reporting items, and European counterparts reported a median of 16.0 (95% CI 14.0–17.2) out of the total of 35 checklist items (Mann Whitney U test, P = 0.048; BF10 = 1.543). European HPGs were better than Croatian HPGs in reporting stakeholder involvement and values and preferences (BF10 = 80.63), as well as describing the implications of costs and resources (BF10 = 55.15). Croatian HPGs better reported HPGs specified aims (BF10 = 16.90), primary intended users (BF10 = 8.70), and sources of funding (BF10 = 122.90). Most insufficiently reported items for both HPG sets were defining the guideline questions and clear outcomes, quality assurance, management of funding and conflicts of interest, and guideline limitations.

Conclusions

Important methodological details are missing from most published HPGs at national and transnational levels. To ensure better quality and adequate use of HPGs, reporting guidelines should be endorsed and used by developers and users alike.
Appendix
Available only for authorised users
Literature
1.
go back to reference Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. In: Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US); 2011. Available from: https://www.ncbi.nlm.nih.gov/books/NBK209539/. doi: 10.17226/13058. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. In: Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US); 2011. Available from: https://​www.​ncbi.​nlm.​nih.​gov/​books/​NBK209539/​.​ doi: 10.17226/13058.
2.
go back to reference Legido-Quigley H, Panteli D, Brusamento S, Knai C, Saliba V, Turk E, et al. Clinical guidelines in the European Union: mapping the regulatory basis, development, quality control, implementation and evaluation across member states. Health policy. 2012;107(2–3):146–56. Epub 2012/09/04CrossRef Legido-Quigley H, Panteli D, Brusamento S, Knai C, Saliba V, Turk E, et al. Clinical guidelines in the European Union: mapping the regulatory basis, development, quality control, implementation and evaluation across member states. Health policy. 2012;107(2–3):146–56. Epub 2012/09/04CrossRef
3.
go back to reference Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76. Epub 2014/01/15CrossRef Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76. Epub 2014/01/15CrossRef
4.
go back to reference Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR network. BMC Med. 2010;8:24. Epub 2010/04/28CrossRef Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR network. BMC Med. 2010;8:24. Epub 2010/04/28CrossRef
5.
go back to reference To MJ, Jones J, Emara M, Jadad AR. Are reports of randomized controlled trials improving over time? A systematic review of 284 articles published in high-impact general and specialized medical journals. PLoS One. 2013;8(12):e84779. Epub 2014/01/07CrossRef To MJ, Jones J, Emara M, Jadad AR. Are reports of randomized controlled trials improving over time? A systematic review of 284 articles published in high-impact general and specialized medical journals. PLoS One. 2013;8(12):e84779. Epub 2014/01/07CrossRef
6.
go back to reference Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB, et al. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology. 2006;67(5):792–7. Epub 2006/09/13CrossRef Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Bossuyt PM, Reitsma JB, et al. The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology. 2006;67(5):792–7. Epub 2006/09/13CrossRef
7.
go back to reference Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001;285(15):1992–5. Epub 2001/04/20CrossRef Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001;285(15):1992–5. Epub 2001/04/20CrossRef
8.
go back to reference Brouwers MC, Kerkvliet K, Spithoff K. The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines. BMJ. 2016;352:i1152. Epub 2016/03/10CrossRef Brouwers MC, Kerkvliet K, Spithoff K. The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines. BMJ. 2016;352:i1152. Epub 2016/03/10CrossRef
9.
go back to reference Chen Y, Yang K, Marusic A, Qaseem A, Meerpohl JJ, Flottorp S, et al. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Intern Med. 2017;166(2):128–32. Epub 2016/11/29CrossRef Chen Y, Yang K, Marusic A, Qaseem A, Meerpohl JJ, Flottorp S, et al. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Intern Med. 2017;166(2):128–32. Epub 2016/11/29CrossRef
10.
go back to reference Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med. 2010;51(5):421–4.CrossRef Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med. 2010;51(5):421–4.CrossRef
11.
go back to reference Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. Epub 2010/03/25CrossRef Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. Epub 2010/03/25CrossRef
12.
go back to reference Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. Epub 2009/07/22CrossRef Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. Epub 2009/07/22CrossRef
13.
go back to reference Chen RB, Chen YL, Gai GZ, Xie YM, Wang YY. Analysis of reporting specification about sepsis practice guideline based on RIGHT standard. Zhongguo Zhong yao za zhi. 2017;42(8):1514–7. Epub 2017/10/27PubMed Chen RB, Chen YL, Gai GZ, Xie YM, Wang YY. Analysis of reporting specification about sepsis practice guideline based on RIGHT standard. Zhongguo Zhong yao za zhi. 2017;42(8):1514–7. Epub 2017/10/27PubMed
14.
go back to reference Xiaoyang S, Lu G, Ru L. Assessment of reporting quality of guidelines on acupuncture and moxibustion by using RIGHT Statement. Chinese Medical Doctor Association Annual Conference on Evidence-based Medicine 2017. Xiaoyang S, Lu G, Ru L. Assessment of reporting quality of guidelines on acupuncture and moxibustion by using RIGHT Statement. Chinese Medical Doctor Association Annual Conference on Evidence-based Medicine 2017.
15.
go back to reference Intersectoral Coordination Mechanism for the Control of Antimicrobial Resistance (ISKRA). Croatian National Guidelines. Available from: http://www.iskra.bfm.hr/eng/Guidlines.aspx?id=61 Intersectoral Coordination Mechanism for the Control of Antimicrobial Resistance (ISKRA). Croatian National Guidelines. Available from: http://​www.​iskra.​bfm.​hr/​eng/​Guidlines.​aspx?​id=​61
17.
go back to reference RIGHT working group. RIGHT Checklist. Available from: http://www.right-statement.org/right-statement/checklist. Accessed 28 May 2018. RIGHT working group. RIGHT Checklist. Available from: http://​www.​right-statement.​org/​right-statement/​checklist.​ Accessed 28 May 2018.
18.
go back to reference Jeffreys H. Theory of probability. Oxford: Oxford University Press; 1961. Jeffreys H. Theory of probability. Oxford: Oxford University Press; 1961.
21.
go back to reference Dykewicz CA. Summary of the guidelines for preventing opportunistic infections among hematopoietic stem cell transplant recipients. Clin Infect Dis. 2001;33(2):139–44.CrossRef Dykewicz CA. Summary of the guidelines for preventing opportunistic infections among hematopoietic stem cell transplant recipients. Clin Infect Dis. 2001;33(2):139–44.CrossRef
22.
go back to reference Guyatt G, Gutterman D, Baumann MH, Addrizzo-Harris D, Hylek EM, Phillips B, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. Chest. 2006;129(1):174–81. Epub 2006/01/21CrossRef Guyatt G, Gutterman D, Baumann MH, Addrizzo-Harris D, Hylek EM, Phillips B, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an american college of chest physicians task force. Chest. 2006;129(1):174–81. Epub 2006/01/21CrossRef
23.
go back to reference SIGN guidelines—an introduction to SIGN methodology for the development of evidence-based clinical guidelines. SIGN Publication No. 39, SIGN Secretariat, Royal College of Physicians of Edinburgh. Edinburgh 1999. SIGN guidelines—an introduction to SIGN methodology for the development of evidence-based clinical guidelines. SIGN Publication No. 39, SIGN Secretariat, Royal College of Physicians of Edinburgh. Edinburgh 1999.
24.
go back to reference Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000;355(9198):103–6. Epub 2000/02/16CrossRef Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet. 2000;355(9198):103–6. Epub 2000/02/16CrossRef
25.
go back to reference Tudor KI, Kozina PN, Marusic A. Methodological rigour and transparency of clinical practice guidelines developed by neurology professional societies in Croatia. PLoS One. 2013;8(7):e69877. Epub 2013/07/31CrossRef Tudor KI, Kozina PN, Marusic A. Methodological rigour and transparency of clinical practice guidelines developed by neurology professional societies in Croatia. PLoS One. 2013;8(7):e69877. Epub 2013/07/31CrossRef
26.
go back to reference ESMO Guidelines Methodology. Available from: http://www.esmo.org/Guidelines/ESMO-Guidelines-Methodology. Accessed 28 May 2018. ESMO Guidelines Methodology. Available from: http://​www.​esmo.​org/​Guidelines/​ESMO-Guidelines-Methodology.​ Accessed 28 May 2018.
27.
go back to reference Wagenmakers EJ, Marsman M, Jamil T, Ly A, Verhagen J, Love J, et al. Bayesian inference for psychology. Part I: theoretical advantages and practical ramifications. Psychon Bull Rev. 2018;25(1):35–57. Epub 2017/08/06CrossRef Wagenmakers EJ, Marsman M, Jamil T, Ly A, Verhagen J, Love J, et al. Bayesian inference for psychology. Part I: theoretical advantages and practical ramifications. Psychon Bull Rev. 2018;25(1):35–57. Epub 2017/08/06CrossRef
28.
go back to reference Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725.CrossRef Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725.CrossRef
29.
go back to reference Brandt L, Vandvik PO, Alonso-Coello P, Akl EA, Thornton J, Rigau D, et al. Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial. BMJ Open. 2017;7(2):e011569.CrossRef Brandt L, Vandvik PO, Alonso-Coello P, Akl EA, Thornton J, Rigau D, et al. Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial. BMJ Open. 2017;7(2):e011569.CrossRef
30.
go back to reference Neumann I, Alonso-Coello P, Vandvik PO, Agoritsas T, Mas G, Akl EA, et al. Do clinicians want recommendations? A multicenter study comparing evidence summaries with and without GRADE recommendations. J Clin Epidemiol. 2018;99:33–40. Epub 2018/03/14CrossRef Neumann I, Alonso-Coello P, Vandvik PO, Agoritsas T, Mas G, Akl EA, et al. Do clinicians want recommendations? A multicenter study comparing evidence summaries with and without GRADE recommendations. J Clin Epidemiol. 2018;99:33–40. Epub 2018/03/14CrossRef
31.
go back to reference Lee D, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of america practice guidelines. Arch Intern Med. 2011;171(1):18–22.CrossRef Lee D, Vielemeyer O. Analysis of overall level of evidence behind infectious diseases society of america practice guidelines. Arch Intern Med. 2011;171(1):18–22.CrossRef
32.
go back to reference Alexander PE, Brito JP, Neumann I, Gionfriddo MR, Bero L, Djulbegovic B, et al. World Health Organization strong recommendations based on low-quality evidence (study quality) are frequent and often inconsistent with GRADE guidance. J Clin Epidemiol. 2016;72:98–106. Epub 2015/01/27CrossRef Alexander PE, Brito JP, Neumann I, Gionfriddo MR, Bero L, Djulbegovic B, et al. World Health Organization strong recommendations based on low-quality evidence (study quality) are frequent and often inconsistent with GRADE guidance. J Clin Epidemiol. 2016;72:98–106. Epub 2015/01/27CrossRef
33.
go back to reference Neumann I, Brignardello-Petersen R, Wiercioch W, Carrasco-Labra A, Cuello C, Akl E, et al. The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels. Implement Sci. 2016;11:93. Epub 2016/07/16CrossRef Neumann I, Brignardello-Petersen R, Wiercioch W, Carrasco-Labra A, Cuello C, Akl E, et al. The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels. Implement Sci. 2016;11:93. Epub 2016/07/16CrossRef
34.
go back to reference Armstrong MJ, Bloom JA. Patient involvement in guidelines is poor five years after institute of medicine standards: review of guideline methodologies. Res Involv Engagem. 2017;3:19. Epub 2017/10/25CrossRef Armstrong MJ, Bloom JA. Patient involvement in guidelines is poor five years after institute of medicine standards: review of guideline methodologies. Res Involv Engagem. 2017;3:19. Epub 2017/10/25CrossRef
35.
go back to reference Chong CA, Chen IJ, Naglie G, Krahn MD. How well do guidelines incorporate evidence on patient preferences? J Gen Intern Med. 2009;24(8):977–82. Epub 2009/04/24CrossRef Chong CA, Chen IJ, Naglie G, Krahn MD. How well do guidelines incorporate evidence on patient preferences? J Gen Intern Med. 2009;24(8):977–82. Epub 2009/04/24CrossRef
36.
go back to reference Dizon JM, Machingaidze S, Grimmer K. To adopt, to adapt, or to contextualise? The big question in clinical practice guideline development. BMC Res Notes. 2016;9(1):442. Epub 2016/09/15CrossRef Dizon JM, Machingaidze S, Grimmer K. To adopt, to adapt, or to contextualise? The big question in clinical practice guideline development. BMC Res Notes. 2016;9(1):442. Epub 2016/09/15CrossRef
37.
go back to reference Gonzalez-Suarez C, Grimmer-Somers K, Dizon J, King E, Lorenzo S, Valdecanas C, et al. Contextualizing Western guidelines for stroke and low back pain to a developing country (Philippines): an innovative approach to putting evidence into practice efficiently. J Healthc Leadersh. 2012;4:141–56.CrossRef Gonzalez-Suarez C, Grimmer-Somers K, Dizon J, King E, Lorenzo S, Valdecanas C, et al. Contextualizing Western guidelines for stroke and low back pain to a developing country (Philippines): an innovative approach to putting evidence into practice efficiently. J Healthc Leadersh. 2012;4:141–56.CrossRef
38.
go back to reference Norris SL, Holmer HK, Ogden LA, Burda BU. Conflict of interest in clinical practice guideline development: a systematic review. PLoS One. 2011;6(10):e25153. Epub 2011/11/01CrossRef Norris SL, Holmer HK, Ogden LA, Burda BU. Conflict of interest in clinical practice guideline development: a systematic review. PLoS One. 2011;6(10):e25153. Epub 2011/11/01CrossRef
39.
go back to reference Wang AT, McCoy CP, Murad MH, Montori VM. Association between industry affiliation and position on cardiovascular risk with rosiglitazone: cross sectional systematic review. BMJ. 2010;340:c1344. Epub 2010/03/20CrossRef Wang AT, McCoy CP, Murad MH, Montori VM. Association between industry affiliation and position on cardiovascular risk with rosiglitazone: cross sectional systematic review. BMJ. 2010;340:c1344. Epub 2010/03/20CrossRef
40.
go back to reference Moynihan RN, Cooke GP, Doust JA, Bero L, Hill S, Glasziou PP. Expanding disease definitions in guidelines and expert panel ties to industry: a cross-sectional study of common conditions in the United States. PLoS Med. 2013;10(8):e1001500. Epub 2013/08/24CrossRef Moynihan RN, Cooke GP, Doust JA, Bero L, Hill S, Glasziou PP. Expanding disease definitions in guidelines and expert panel ties to industry: a cross-sectional study of common conditions in the United States. PLoS Med. 2013;10(8):e1001500. Epub 2013/08/24CrossRef
42.
go back to reference Rasmussen K, Schroll J, Gotzsche PC, Lundh A. Under-reporting of conflicts of interest among trialists: a cross-sectional study. J R Soc Med. 2015;108(3):101–7. Epub 2014/11/13CrossRef Rasmussen K, Schroll J, Gotzsche PC, Lundh A. Under-reporting of conflicts of interest among trialists: a cross-sectional study. J R Soc Med. 2015;108(3):101–7. Epub 2014/11/13CrossRef
44.
go back to reference Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ. 2008;336(7652):1049–51.CrossRef Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ. 2008;336(7652):1049–51.CrossRef
45.
go back to reference Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123–42. Epub 2013/12/18CrossRef Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123–42. Epub 2013/12/18CrossRef
46.
go back to reference Grimmer K, Dizon JM, Milanese S, King E, Beaton K, Thorpe O, et al. Efficient clinical evaluation of guideline quality: development and testing of a new tool. BMC Med Res Methodol. 2014;14:63. Epub 2014/06/03CrossRef Grimmer K, Dizon JM, Milanese S, King E, Beaton K, Thorpe O, et al. Efficient clinical evaluation of guideline quality: development and testing of a new tool. BMC Med Res Methodol. 2014;14:63. Epub 2014/06/03CrossRef
47.
go back to reference Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brozek J, et al. Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013;8:6. Epub 2013/01/11CrossRef Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brozek J, et al. Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013;8:6. Epub 2013/01/11CrossRef
Metadata
Title
Reporting quality of European and Croatian health practice guidelines according to the RIGHT reporting checklist
Authors
Ružica Tokalić
Marin Viđak
Ivan Buljan
Ana Marušić
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2018
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-018-0828-4

Other articles of this Issue 1/2018

Implementation Science 1/2018 Go to the issue