Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2018

Open Access 01-01-2018 | Method

Applying GRADE-CERQual to qualitative evidence synthesis findings–paper 7: understanding the potential impacts of dissemination bias

Authors: Andrew Booth, Simon Lewin, Claire Glenton, Heather Munthe-Kaas, Ingrid Toews, Jane Noyes, Arash Rashidian, Rigmor C. Berg, Brenda Nyakang’o, Joerg J. Meerpohl, GRADE-CERQual Coordinating Team

Published in: Implementation Science | Special Issue 1/2018

Login to get access

Abstract

Background

The GRADE-CERQual (Confidence in Evidence from Reviews of Qualitative research) approach has been developed by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group. The approach has been developed to support the use of findings from qualitative evidence syntheses in decision-making, including guideline development and policy formulation.
CERQual includes four components for assessing how much confidence to place in findings from reviews of qualitative research (also referred to as qualitative evidence syntheses): (1) methodological limitations, (2) coherence, (3) adequacy of data and (4) relevance. This paper is part of a series providing guidance on how to apply CERQual and focuses on a probable fifth component, dissemination bias. Given its exploratory nature, we are not yet able to provide guidance on applying this potential component of the CERQual approach. Instead, we focus on how dissemination bias might be conceptualised in the context of qualitative research and the potential impact dissemination bias might have on an overall assessment of confidence in a review finding. We also set out a proposed research agenda in this area.

Methods

We developed this paper by gathering feedback from relevant research communities, searching MEDLINE and Web of Science to identify and characterise the existing literature discussing or assessing dissemination bias in qualitative research and its wider implications, developing consensus through project group meetings, and conducting an online survey of the extent, awareness and perceptions of dissemination bias in qualitative research.

Results

We have defined dissemination bias in qualitative research as a systematic distortion of the phenomenon of interest due to selective dissemination of studies or individual study findings. Dissemination bias is important for qualitative evidence syntheses as the selective dissemination of qualitative studies and/or study findings may distort our understanding of the phenomena that these syntheses aim to explore and thereby undermine our confidence in these findings.
Dissemination bias has been extensively examined in the context of randomised controlled trials and systematic reviews of such studies. The effects of potential dissemination bias are formally considered, as publication bias, within the GRADE approach. However, the issue has received almost no attention in the context of qualitative research. Because of very limited understanding of dissemination bias and its potential impact on review findings in the context of qualitative evidence syntheses, this component is currently not included in the GRADE-CERQual approach.

Conclusions

Further research is needed to establish the extent and impacts of dissemination bias in qualitative research and the extent to which dissemination bias needs to be taken into account when we assess how much confidence we have in findings from qualitative evidence syntheses.
Appendix
Available only for authorised users
Literature
1.
Toews I, Booth A, Berg RC, Lewin S, Glenton C, Munthe-Kaas HM, Noyes J, Schroter S, Meerpohl JJ. Further exploration of dissemination bias in qualitative research required to facilitate assessment within qualitative evidence syntheses. J Clin Epidemiol. 2017;88:133–9.CrossRefPubMed
2.
Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gulmezoglu M, Noyes J, Booth A, Garside R, Rashidian A. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med. 2015;12(10):e1001895.
3.
Toews I, Glenton C, Lewin S, Berg RC, Noyes J, Booth A, Marusic A, Malicki M, Munte-Kaas HM, Meerpohl JJ. Extent, awareness and perception of dissemination bias in qualitative research: an explorative survey. PLoS One. 2016;11(8):e0159290.CrossRefPubMedPubMedCentral
4.
Lewin S, Booth A, Glenton C, Munthe-Kaas HM, Rashidian A, Wainwright M, Bohren MA, Tuncalp Ö, Colvin CJ, Garside R, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. 2018;13(Suppl 1): https://​doi.​org/​10.​1186/​s13012-017-0688-3.
5.
Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3(8):e3081.CrossRefPubMedPubMedCentral
6.
Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, Williamson PR, Kirkham JJ. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11(6):e1001666.CrossRefPubMedPubMedCentral
7.
Schmucker C, Schell LK, Portalupi S, Oeller P, Cabrera L, Bassler D, Schwarzer G, Scherer RW, Antes G, von Elm E, et al. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One. 2014;9(12):e114023.CrossRefPubMedPubMedCentral
8.
Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, Alonso-Coello P, Djulbegovic B, Atkins D, Falck-Ytter Y, et al. GRADE guidelines: 5. Rating the quality of evidence––publication bias. J Clin Epidemiol. 2011;64(12):1277–82.CrossRefPubMed
9.
Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, Hing C, Kwok CS, Pang C, Harvey I. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):iii. ix-xi, 1-193CrossRefPubMed
10.
Hammersley M. What’s wrong with ethnography? ––methodological explorations. London: Routledge; 1992.
11.
12.
Petticrew M, Egan M, Thomson H, Hamilton V, Kunkler R, Roberts H. Publication bias in qualitative research: what becomes of qualitative research presented at conferences? J Epidemiol Community Health. 2008;62(6):552–4.CrossRefPubMed
13.
14.
Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, Al-Shahi Salman R, Chan AW, Glasziou P. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101–4.CrossRefPubMed
15.
Campbell R, Pound P, Morgan M, Daker-White G, Britten N, Pill R, Yardley L, Pope C, Donovan J. Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technol Assess. 2011;15(43):1–164.CrossRefPubMed
16.
Dickson-Swift V, James EL, Kippen S, Liamputtong P. Researching sensitive topics: qualitative research as emotion work. Qual Res. 2009;9(1):61–79.CrossRef
17.
Bassler D, Mueller KF, Briel M, Kleijnen J, Marusic A, Wager E, Antes G, von Elm E, Altman DG, Meerpohl JJ, et al. Bias in dissemination of clinical research findings: structured OPEN framework of what, who and why, based on literature review and expert consensus. BMJ Open. 2016;6(1):e010024.CrossRefPubMedPubMedCentral
18.
Sterne JAC, Egger M, Moher D. Chapter 10: addressing reporting biases. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Intervention Version 510 (updated March 2011). The Cochrane Collaboration; 2011. Available from http://​www.​cochrane-handbook.​org/​. Accessed 20 Nov 2017.
19.
Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;2:MR000005.
20.
Scherer RW, Ugarte-Gil C, Schmucker C, Meerpohl JJ. Authors report lack of time as main reason for unpublished research presented at biomedical conferences: a systematic review. J Clin Epidemiol. 2015;68(7):803–10.CrossRefPubMedPubMedCentral
21.
Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, Shaw RL, Smith JA, Young B. How can systematic reviews incorporate qualitative research? A critical perspective. Qual Res. 2006;6(1):27–44.CrossRef
22.
Booth A, Carroll C, Ilott I, Low LL, Cooper K. Desperately seeking dissonance: identifying the disconfirming case in qualitative evidence synthesis. Qual Health Res. 2013;23(1):126–41.CrossRefPubMed
23.
Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J. Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol. 2008;8:21.CrossRefPubMedPubMedCentral
24.
Newton BJ, Rothlingova Z, Gutteridge R, LeMarchand K, Raphael JH. No room for reflexivity? Critical reflections following a systematic review of qualitative research. J Health Psychol. 2012;17(6):866–85.CrossRefPubMed
25.
Ferrer HB, Trotter C, Hickman M, Audrey S. Barriers and facilitators to HPV vaccination of young women in high-income countries: a qualitative systematic review and evidence synthesis. BMC Public Health. 2014;14:700.CrossRefPubMedPubMedCentral
26.
Cosco TD, Prina AM, Perales J, Stephan BC, Brayne C. Lay perspectives of successful ageing: a systematic review and meta-ethnography. BMJ Open. 2013;3(6).
27.
Fu Y, McNichol E, Marczewski K, Closs SJ. Patient-professional partnerships and chronic back pain self-management: a qualitative systematic review and synthesis. Health Soc Care Community. 2015.
28.
Gandhi G. Charting the evolution of approaches employed by the Global Alliance for Vaccines and mmunizations (GAVI) to address inequities in access to immunization: a systematic qualitative review of GAVI policies, strategies and resource allocation mechanisms through an equity lens (1999–2014). BMC Public Health. 2015;15:1198.
29.
Mills EJ, Montori VM, Ross CP, Shea B, Wilson K, Guyatt GH. Systematically reviewing qualitative studies complements survey design: an exploratory study of barriers to paediatric immunisations. J Clin Epidemiol. 2005;58(11):1101–8.CrossRefPubMed
30.
Satink T, Cup EH, Ilott I, Prins J, de Swart BJ, Nijhuis-van der Sanden MW. Patients’ views on the impact of stroke on their roles and self: a thematic synthesis of qualitative studies. Arch Phys Med Rehabil. 2013;94(6):1171–83.CrossRefPubMed
31.
Metadata
Title
Applying GRADE-CERQual to qualitative evidence synthesis findings–paper 7: understanding the potential impacts of dissemination bias
Authors
Andrew Booth
Simon Lewin
Claire Glenton
Heather Munthe-Kaas
Ingrid Toews
Jane Noyes
Arash Rashidian
Rigmor C. Berg
Brenda Nyakang’o
Joerg J. Meerpohl
GRADE-CERQual Coordinating Team
Publication date
01-01-2018
Publisher
BioMed Central
Published in
Implementation Science / Issue Special Issue 1/2018
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-017-0694-5

Other articles of this Special Issue 1/2018

Implementation Science 1/2018 Go to the issue

Introduction

Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 4: how to assess coherence

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 3: how to assess methodological limitations

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 5: how to assess adequacy of data

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 2: how to make an overall CERQual assessment of confidence and create a Summary of Qualitative Findings table

Method

Applying GRADE-CERQual to qualitative evidence synthesis findings–paper 6: how to assess relevance of the data