Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2018

Open Access 01-01-2018 | Introduction

Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series

Authors: Simon Lewin, Andrew Booth, Claire Glenton, Heather Munthe-Kaas, Arash Rashidian, Megan Wainwright, Meghan A. Bohren, Özge Tunçalp, Christopher J. Colvin, Ruth Garside, Benedicte Carlsen, Etienne V. Langlois, Jane Noyes

Published in: Implementation Science | Special Issue 1/2018

Login to get access

Abstract

The GRADE-CERQual (‘Confidence in the Evidence from Reviews of Qualitative research’) approach provides guidance for assessing how much confidence to place in findings from systematic reviews of qualitative research (or qualitative evidence syntheses). The approach has been developed to support the use of findings from qualitative evidence syntheses in decision-making, including guideline development and policy formulation. Confidence in the evidence from qualitative evidence syntheses is an assessment of the extent to which a review finding is a reasonable representation of the phenomenon of interest. CERQual provides a systematic and transparent framework for assessing confidence in individual review findings, based on consideration of four components: (1) methodological limitations, (2) coherence, (3) adequacy of data, and (4) relevance. A fifth component, dissemination (or publication) bias, may also be important and is being explored. As with the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach for effectiveness evidence, CERQual suggests summarising evidence in succinct, transparent, and informative Summary of Qualitative Findings tables. These tables are designed to communicate the review findings and the CERQual assessment of confidence in each finding. This article is the first of a seven-part series providing guidance on how to apply the CERQual approach. In this paper, we describe the rationale and conceptual basis for CERQual, the aims of the approach, how the approach was developed, and its main components. We also outline the purpose and structure of this series and discuss the growing role for qualitative evidence in decision-making. Papers 3, 4, 5, 6, and 7 in this series discuss each CERQual component, including the rationale for including the component in the approach, how the component is conceptualised, and how it should be assessed. Paper 2 discusses how to make an overall assessment of confidence in a review finding and how to create a Summary of Qualitative Findings table. The series is intended primarily for those undertaking qualitative evidence syntheses or using their findings in decision-making processes but is also relevant to guideline development agencies, primary qualitative researchers, and implementation scientists and practitioners.
Appendix
Available only for authorised users
Literature
1.
Oxman AD, Lavis JN, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 1: what is evidence-informed policymaking? Health Res Policy Syst. 2009;7(Suppl 1):S1.CrossRefPubMedPubMedCentral
2.
Nabyonga-Orem J, Mijumbi R. Evidence for informing health policy development in low-income countries (LICs): perspectives of policy actors in Uganda. Int J Health Policy Manag. 2015;4(5):285–93.CrossRefPubMedPubMedCentral
3.
Rashidian A, Eccles MP, Russell I. Falling on stony ground? A qualitative study of implementation of clinical guidelines’ prescribing recommendations in primary care. Health Policy. 2008;85(2):148–61.CrossRefPubMed
4.
Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brozek J, Davoli M, Flottorp S, Harbour R, Hill S, et al. Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013;8:6.CrossRefPubMedPubMedCentral
5.
Davies P: Evidence-based government: how can we make it happen? CHSRF (Canada Health Services Research Foundation) 7th annual invitational workshop - leveraging knowledge: tools and strategies for action. Montreal, Canada; 2005.
6.
Colvin CJ, de Heer J, Winterton L, Mellenkamp M, Glenton C, Noyes J, Lewin S, Rashidian A. A systematic review of qualitative evidence on barriers and facilitators to the implementation of task-shifting in midwifery services. Midwifery. 2013;29(10):1211–21.CrossRefPubMed
7.
Glenton C, Colvin CJ, Carlsen B, Swartz A, Lewin S, Noyes J, Rashidian A. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database Syst Rev. 2013;10:CD010414.
8.
Rashidian A, Shakibazadeh E, Karimi-Shahanjarini A, Glenton C, Noyes J, Lewin S, Colvin C, Laurant M. Barriers and facilitators to the implementation of doctor-nurse substitution strategies in primary care: qualitative evidence synthesis (protocol). Cochrane Database Syst Rev. 2013;2:CD010412.
9.
Leach-Kemon K, Chou DP, Schneider MT, Tardif A, Dieleman JL, Brooks BP, Hanlon M, Murray CJ. The global financial crisis has led to a slowdown in growth of funding to improve health in many developing countries. Health Aff (Millwood). 2012;31(1):228–35.CrossRef
10.
Gough D, Oliver S, Thomas J. An introduction to systematic reviews. Thousand Oaks: Sage Publications; 2012.
11.
Noyes J, Popay J, Pearson A, Hannes K, Booth A, Cochrane Qualitative Research Methods Group. Chapter 20: Qualitative research and Cochrane reviews. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 510 [updated March 2011]: The Cochrane Collaboration; 2011. Available from: http://​handbook-5-1.​cochrane.​org/​
12.
Saini M, Shlonsky A. Systematic synthesis of qualitative research. Oxford, UK: Oxford University Press; 2012.CrossRef
13.
Ansari S, Rashidian A. Guidelines for guidelines: are they up to the task? A comparative assessment of clinical practice guideline development handbooks. PLoS One. 2012;7(11):e49864.CrossRefPubMedPubMedCentral
14.
15.
Lewin S, Bosch-Capblanch X, Oliver S, Akl EA, Vist GE, Lavis J, Ghersi D, Røttingen J, Steinmann P, Gulmezoglu M, et al. Guidance for evidence-informed policies about health systems: assessing how much confidence to place in the research evidence. PLoS Med. 2012;9(3):e1001187.CrossRefPubMedPubMedCentral
16.
Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gulmezoglu M, Noyes J, Booth A, Garside R, Rashidian A. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med. 2015;12(10):e1001895.CrossRefPubMedPubMedCentral
17.
Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.CrossRefPubMed
18.
Hsu J, Brozek JL, Terracciano L, Kreis J, Compalati E, Stein AT, Fiocchi A, Schunemann HJ. Application of GRADE: making evidence-based recommendations about diagnostic tests in clinical practice guidelines. Implement Sci. 2011;6:62.CrossRefPubMedPubMedCentral
19.
Sandelowski M, Docherty S, Emden C. Focus on qualitative methods qualitative metasynthesis: issues and techniques. Res Nurs Health. 1997;20:365–72.CrossRefPubMed
20.
Thorne S. Metasynthetic madness: what kind of monster have we created? Qual Health Res. 2017;27(1):3–13.CrossRefPubMed
21.
Thorne S, Jensen L, Kearney M, Noblit G, Sandelowski M. Reflections on the methodological and ideological agenda in qualitative meta-synthesis. Qual Health Res. 2004;14:1342–65.CrossRefPubMed
22.
Hammersley M. What’s wrong with ethnography?—methodological explorations. London: Routledge; 1992.
23.
Barth F. Analytical dimensions in the comparison of social organizations. Am Anthropol. 1972;74(1-2):207–20.CrossRef
24.
Pope C, Mays N, Popay J. Synthesising qualitative and quantitative health evidence: a guide to methods. United Kingdom: McGraw-Hill Education; 2007.
25.
Lewin S, Bohren M, Rashidian A, Glenton C, Munthe-Kaas HM, Carlsen B, Colvin CJ, Tuncalp Ö, Noyes J, Booth A et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table. Implement Sci. 2018;13(Suppl 1). https://​doi.​org/​10.​1186/​s13012-017-0689-2.
26.
Ames HMR, Glenton C, Lewin S. Parents’ and informal caregivers’ views and experiences of communication about routine childhood vaccination: a synthesis of qualitative evidence. Cochrane Database Syst Rev. 2017;2:CD011787.PubMed
27.
Aslam RW, Hendry M, Carter B, Noyes J, Rycroft Malone J, Booth A, Pasterfield D, Charles JM, Craine N, Tudor Edwards R et al: Interventions for preventing unintended repeat pregnancies among adolescents (Protocol). Cochrane Database of Systematic Reviews. 2015(1):Art. No.: CD011477.
28.
Bohren MA, Hunter EC, Munthe-Kaas HM, Souza JP, Vogel JP, Gulmezoglu AM. Facilitators and barriers to facility-based delivery in low- and middle-income countries: a qualitative evidence synthesis. Reprod Health. 2014;11(1):71.CrossRefPubMedPubMedCentral
29.
Bohren MA, Vogel JP, Hunter EC, Lutsiv O, Makh SK, Souza JP, Aguiar C, Saraiva Coneglian F, Diniz AL, Tuncalp O, et al. The mistreatment of women during childbirth in health facilities globally: a mixed-methods systematic review. PLoS Med. 2015;12(6):e1001847. discussion e1001847CrossRefPubMedPubMedCentral
30.
Munabi-Babigumira S, Glenton C, Lewin S, Fretheim A, Nabudere H. Factors that influence the provision of intrapartum and postnatal care by skilled birth attendants in low- and middle-income countries: a qualitative evidence synthesis. Cochrane Database Syst Rev. 2017;(11):CD011558.
31.
Munthe-Kaas HM, Hammerstrøm KT, et al. Effekt av og erfaringer med kontinuitetsfremmende tiltak i barnevernsinstitusjoner. Norwegian Knowledge Centre for the Health Services: Oslo; 2013.
32.
O'Brien TD, Noyes J, Spencer LH, Kubis HP, Hastings RP, Edwards RT, Bray N, Whitaker R. Keep fit exercise interventions to improve health, fitness and well-being of children and young people who use wheelchairs: mixed-method systematic review protocol. J Adv Nurs. 2014;70(12):2942–51.CrossRefPubMed
33.
Whitaker R, Hendry M, Booth A, Carter B, Charles J, Craine N, Edwards RT, Lyons M, Noyes J, Pasterfield D, et al. Intervention Now To Eliminate Repeat Unintended Pregnancy in Teenagers (INTERUPT): a systematic review of intervention effectiveness and cost-effectiveness, qualitative and realist synthesis of implementation factors and user engagement. BMJ Open. 2014;4(4):e004733.CrossRefPubMedPubMedCentral
34.
Downe S, Finlayson K, Tuncalp Ӧ, Metin Gulmezoglu A. What matters to women: a systematic scoping review to identify the processes and outcomes of antenatal care provision that are important to healthy pregnant women. BJOG. 2016;123(4):529–39.CrossRefPubMed
35.
Odendaal WA, Goudge J, Griffiths F, Tomlinson M, Leon N, Daniels K: Healthcare workers’ perceptions and experience on using mHealth technologies to deliver primary healthcare services: qualitative evidence synthesis (Protocol). Cochrane Database of Systematic Reviews 2015(11):Art. No.: CD011942.
36.
37.
Booth A, Lewin S, Glenton C, Munthe-Kaas HM, Meerpohl J, Rees R, Noyes J, Rashidian A, Berg R, Nyakango B, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 7: understanding the potential impacts of dissemination bias. Implement Sci. 2018;13(Suppl 1). https://​doi.​org/​10.​1186/​s13012-017-0694-5.
38.
Toews I, Booth A, Berg RC, Lewin S, Glenton C, Munthe-Kaas HM, Noyes J, Schroter S, Meerpohl JJ. Further exploration of dissemination bias in qualitative research required to facilitate assessment within qualitative evidence syntheses. J Clin Epidemiol. 2017;88:133–39.
39.
Toews I, Glenton C, Lewin S, Berg RC, Noyes J, Booth A, Marusic A, Malicki M, Munte-Kaas HM, Meerpohl JJ. Extent, awareness and perception of dissemination bias in qualitative research: an explorative survey. PLoS One. 2016;11(8):e0159290.CrossRefPubMedPubMedCentral
40.
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ, Group GW. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6.CrossRefPubMedPubMedCentral
41.
42.
Booth A, Noyes J, Flemming K, Gerhardus A, Wahlster P, Van Der Wilt GJ, Mozygemba K, Refolo P, Sacchini D, Tummers M et al: Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions [online]. Available from: http://​www.​integrate-hta.​eu/​downloads/​ 2016.
43.
Colvin CJ, Garside R, Wainwright M, Lewin S, Bohren M, Glenton C, Munthe-Kaas HM, Carlsen B, Tuncalp Ö, Noyes J, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 4: how to assess coherence. Implement Sci. 2018;13(Suppl 1). https://​doi.​org/​10.​1186/​s13012-017-0691-8.
44.
Glenton C, Carlsen B, Lewin S, Munthe-Kaas HM, Colvin CJ, Tuncalp Ö, Bohren M, Noyes J, Booth A, Garside R, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 5: how to assess adequacy of data. Implement Sci. 2018;13(Suppl 1). https://​doi.​org/​10.​1186/​s13012-017-0692-7.
45.
Munthe-Kaas HM, Bohren M, Carlsen B, Glenton C, Lewin S, Colvin CJ, Tuncalp Ö, Noyes J, Booth A, Garside R, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 3: how to assess methodological limitations. Implement Sci. 2018;13(Suppl 1). https://​doi.​org/​10.​1186/​s13012-017-0690-9.
46.
Noyes J, Booth A, Lewin S, Carlsen B, Glenton C, munthe-Kaas HM, Colvin CJ, Garside R, Bohren M, Rashidian A, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 6: how to assess relevance of the data. Implement Sci. 2018;13(Suppl 1). https://​doi.​org/​10.​1186/​s13012-017-0693-6.
47.
Glenton C, Lewin S, Norris SL. Using evidence from qualitative research to develop WHO guidelines (Chapter 15). In: World Health Organization, editor. Handbook for Guideline Development. 2nd ed. Geneva: WHO; 2016.
48.
Oxman AD, Lewin S, Lavis JN, Fretheim A. SUPPORT tools for evidence-informed health policymaking (STP) 15: engaging the public in evidence-informed policymaking. Health Syst Policy Res. 2009;7(Suppl 1):S15.CrossRef
49.
Abelson J, Blacksher EA, Li KK, Boesveld SE, Goold SD. Public deliberation in health policy and bioethics: mapping an emerging, interdisciplinary field. J Public Deliberation. 2013;9(1)
50.
Davies C, Wetherell M, Barnett E. Citizens at the centre: deliberative participation in healthcare decisions. Bristol: Policy Press; 2006.
51.
Street J, Duszynski K, Krawczyk S, Braunack-Mayer A. The use of citizens’ juries in health policy decision-making: a systematic review. Soc Sci Med. 2014;109:1–9.CrossRefPubMed
52.
McCoy DC, Hall JA, Ridge M. A systematic review of the literature for evidence on health facility committees in low- and middle-income countries. Health Policy Plan. 2012;27(6):449–66.CrossRefPubMed
53.
Molyneux S, Atela M, Angwenyi V, Goodman C. Community accountability at peripheral health facilities: a review of the empirical literature and development of a conceptual framework. Health Policy Plan. 2012;27(7):541–54.CrossRefPubMedPubMedCentral
54.
NICE. Community engagement: improving health and wellbeing and reducing health inequalities. NICE Guideline. London, UK: National Institute for Health and Care Excellence (NICE); 2016.
55.
Lavis JN, Boyko JA, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health Policymaking (STP) 14: organising and using policy dialogues to support evidence-informed policymaking. Health Res Policy Syst. 2009;7(Suppl 1):S14.CrossRefPubMedPubMedCentral
56.
WHO. Handbook for guideline development (2nd edition). Geneva: World Health Organization; 2016.
57.
58.
Metadata
Title
Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series
Authors
Simon Lewin
Andrew Booth
Claire Glenton
Heather Munthe-Kaas
Arash Rashidian
Megan Wainwright
Meghan A. Bohren
Özge Tunçalp
Christopher J. Colvin
Ruth Garside
Benedicte Carlsen
Etienne V. Langlois
Jane Noyes
Publication date
01-01-2018
Publisher
BioMed Central
Published in
Implementation Science / Issue Special Issue 1/2018
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-017-0688-3

Other articles of this Special Issue 1/2018

Implementation Science 1/2018 Go to the issue

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings–paper 7: understanding the potential impacts of dissemination bias

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 5: how to assess adequacy of data

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 3: how to assess methodological limitations

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings–paper 6: how to assess relevance of the data

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 4: how to assess coherence

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table