Top

Systematic Reviews

Published in:

Open Access 01-12-2017 | Research

Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence

Authors: Quan Nha Hong, Pierre Pluye, Mathieu Bujold, Maggy Wassef

Published in: Systematic Reviews | Issue 1/2017

Abstract

Background

Systematic reviews of qualitative and quantitative evidence can provide a rich understanding of complex phenomena. This type of review is increasingly popular, has been used to provide a landscape of existing knowledge, and addresses the types of questions not usually covered in reviews relying solely on either quantitative or qualitative evidence. Although several typologies of synthesis designs have been developed, none have been tested on a large sample of reviews. The aim of this review of reviews was to identify and develop a typology of synthesis designs and methods that have been used and to propose strategies for synthesizing qualitative and quantitative evidence.

Methods

A review of systematic reviews combining qualitative and quantitative evidence was performed. Six databases were searched from inception to December 2014. Reviews were included if they were systematic reviews combining qualitative and quantitative evidence. The included reviews were analyzed according to three concepts of synthesis processes: (a) synthesis methods, (b) sequence of data synthesis, and (c) integration of data and synthesis results.

Results

A total of 459 reviews were included. The analysis of this literature highlighted a lack of transparency in reporting how evidence was synthesized and a lack of consistency in the terminology used. Two main types of synthesis designs were identified: convergent and sequential synthesis designs. Within the convergent synthesis design, three subtypes were found: (a) data-based convergent synthesis design, where qualitative and quantitative evidence is analyzed together using the same synthesis method, (b) results-based convergent synthesis design, where qualitative and quantitative evidence is analyzed separately using different synthesis methods and results of both syntheses are integrated during a final synthesis, and (c) parallel-results convergent synthesis design consisting of independent syntheses of qualitative and quantitative evidence and an interpretation of the results in the discussion.

Conclusions

Performing systematic reviews of qualitative and quantitative evidence is challenging because of the multiple synthesis options. The findings provide guidance on how to combine qualitative and quantitative evidence. Also, recommendations are made to improve the conducting and reporting of this type of review.

Bunn F, Trivedi D, Alderson P, Hamilton L, Martin A, Pinkney E, et al. The impact of Cochrane Reviews: a mixed-methods evaluation of outputs from Cochrane Review Groups supported by the National Institute for Health Research. Health Technol Assess. 2015;19(28):1–100. doi:10.1186/2046-4053-3-125.CrossRefPubMedPubMedCentral

Goldsmith MR, Bankhead CR, Austoker J. Synthesising quantitative and qualitative research in evidence-based patient information. J Epidemiol Community Health. 2007;61(3):262–70. doi:10.1136/jech.2006.046110.CrossRefPubMedPubMedCentral

Victora CG, Habicht J-P, Bryce J. Evidence-based public health: moving beyond randomized trials. Am J Public Health. 2004;94(3):400–5. doi:10.2105/AJPH.94.3.400.CrossRefPubMedPubMedCentral

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Integrative approaches to qualitative and quantitative evidence. London, UK: Health Development Agency; 2004. Report No.: 1842792555.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546–53. doi:10.1111/j.1365-2648.2005.03621.x.CrossRefPubMed

Harden A, Thomas J. Methodological issues in combining diverse study types in systematic reviews. Int J Soc Res Meth. 2005;8(3):257–71. doi:10.1080/13645570500155078.CrossRef

Heyvaert M, Hannes K, Onghena P. Using mixed methods research synthesis for literature reviews: the mixed methods research synthesis approach. Thousand Oaks, CA: SAGE Publications; 2016.

Sandelowski M, Voils CI, Barroso J. Defining and designing mixed research synthesis studies. Res Schools. 2006;13(1):29–40.

Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud. 2009;46(4):529–46. doi:10.1016/j.ijnurstu.2009.01.009.CrossRefPubMed

10.

Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014;35:29–45. doi:10.1146/annurev-publhealth-032013-182440.CrossRefPubMed

11.

Crowe M, Sheppard L. A review of critical appraisal tools show they lack rigor: alternative tool structure is proposed. J Clin Epidemiol. 2011;64(1):79–89. doi:10.1016/j.jclinepi.2010.02.008.CrossRefPubMed

12.

Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract. 2012;18(4):746–52. doi:10.1111/j.1365-2753.2011.01662.x.CrossRefPubMed

13.

Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005;10 Suppl 1:6–20. doi:10.1258/1355819054308576.CrossRefPubMed

14.

Tricco AC, Antony J, Soobiah C, Kastner M, MacDonald H, Cogo E, et al. Knowledge synthesis methods for integrating qualitative and quantitative data: a scoping review reveals poor operationalization of the methodological steps. J Clin Epidemiol. 2016;73:29–35. doi:10.1016/j.jclinepi.2015.12.011.CrossRefPubMed

15.

Plano Clark VL, Ivankova NV. Mixed methods research: a guide to the field. SAGE mixed methods research series. Thousand Oaks: SAGE Publications; 2015.

16.

Langley A. Strategies for theorizing from process data. Acad Manage Rev. 1999;24(4):691–710. doi:10.2307/259349.

17.

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 2016;104(3):240. doi:10.3163/1536-5050.104.3.014.CrossRefPubMedPubMedCentral

18.

Booth A, Papaioannou D, Sutton A. Systematic approaches to a successful literature review. London: SAGE Publications; 2012.

19.

Whiting P, Savovic J, Higgins JP, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34. doi:10.1016/j.jclinepi.2015.06.005.CrossRefPubMedPubMedCentral

20.

Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10):1013–20. doi:10.1016/j.jclinepi.2008.10.009.CrossRefPubMed

21.

Weber M, Freund J, Kamnitzer P, Bertrand P. Économie et société: les catégories de la sociologie. Paris: Pocket; 1995.

22.

Heyvaert M, Maes B, Onghena P. Mixed methods research synthesis: definition, framework, and potential. Qual Quant. 2013;47(2):659–76. doi:10.1007/s11135-011-9538-6.CrossRef

23.

Creswell JW, Plano CV. Designing and conducting mixed methods research. 2nd ed. Thousand Oaks: SAGE Publications; 2011.

24.

Collins KM, O’cathain A. Introduction: ten points about mixed methods research to be considered by the novice researcher. Int J Mult Res Approaches. 2009;3(1):2–7. doi:10.5172/mra.455.3.1.2.CrossRef

25.

Sutton AJ, Jones DR, Abrams KR, Sheldon TA, Song F. Systematic reviews and meta-analysis: a structured review of the methodological literature. J Health Serv Res Policy. 1999;4(1):49–55. doi:10.1177/135581969900400112.CrossRefPubMed

26.

Porta MS, Greenland S, Hernán M, dos Santos SI, Last JM. A dictionary of epidemiology. New York: Oxford University Press; 2014.CrossRef

27.

Neuendorf KA. The content analysis guidebook. Thousand Oaks: SAGE Publications; 2002.

28.

Krippendorff K. Content analysis: an introduction to its methodology. Thousand Oaks: SAGE Publications; 2012.

29.

Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88. doi:10.1177/1049732305276687.CrossRefPubMed

30.

Rihoux B, Marx A. QCA, 25 years after “the comparative method”: mapping, challenges, and innovations—mini-symposium. Polit Res Q. 2013;66(1):167–235. doi:10.1177/1065912912468269.CrossRef

31.

Hedges LV, Olkin I. Vote-counting methods in research synthesis. Psychol Bull. 1980;88(2):359. doi:10.1037/0033-2909.88.2.359.CrossRef

32.

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9(59). doi:10.1186/1471-2288-9-59.

33.

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005;10(1):45–53. doi:10.1258/1355819052801804.CrossRefPubMed

34.

Kastner M, Tricco AC, Soobiah C, Lillie E, Perrier L, Horsley T et al. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Med Res Methodol. 2012;12(114). doi:10.1186/1471-2288-12-114.

35.

Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs—principles and practices. Health Serv Res. 2013;48(6pt2):2134–56. doi:10.1111/1475-6773.12117.CrossRefPubMedPubMedCentral

36.

Guetterman TC, Fetters MD, Creswell JW. Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med. 2015;13(6):554–61. doi:10.1370/afm.1865.

37.

Mills EJ, Nachega JB, Bangsberg DR, Singh S, Rachlis B, Wu P, et al. Adherence to HAART: a systematic review of developed and developing nation patient-reported barriers and facilitators. PLoS Med. 2006;3(11):e438. doi:10.1371/journal.pmed.0030438.CrossRefPubMedPubMedCentral

38.

Langley A, Mintzberg H, Pitcher P, Posada E, Saint-Macary J. Opening up decision making: the view from the black stool. Organ Sci. 1995;6(3):260–79. doi:10.1287/orsc.6.3.260.CrossRef

39.

Van de Ven AH. Suggestions for studying strategy process: a research note. Strat Manag J. 1992;13(5):169–88. doi:10.1002/smj.4250131013.

40.

Frantzen KK, Fetters MD. Meta-integration for synthesizing data in a systematic mixed studies review: insights from research on autism spectrum disorder. Qual Quant. 2015:1-27. doi:10.1007/s11135-015-0261-6.

41.

Whittemore R, Chao A, Jang M, Minges KE, Park C. Methods for knowledge synthesis: an overview. Heart Lung. 2014;43(5):453–61. doi:10.1016/j.hrtlng.2014.05.014.CrossRefPubMed

42.

O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–8. doi:10.1258/jhsrp.2007.007074.CrossRefPubMed

43.

Tricco AC, Soobiah C, Antony J, Cogo E, MacDonald H, Lillie E, et al. A scoping review identifies multiple emerging knowledge synthesis methods, but few studies operationalize the method. J Clin Epidemiol. 2016;73:19–28. doi:10.1016/j.jclinepi.2015.08.030.CrossRefPubMed

44.

Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews. Lancaster, UK: Lancaster University; 2006.

45.

Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev. 2012;1(28). doi:10.1186/2046-4053-1-28.

46.

Anderson LM, Oliver SR, Michie S, Rehfuess E, Noyes J, Shemilt I. Investigating complexity in systematic reviews of interventions by using a spectrum of methods. J Clin Epidemiol. 2013;66(11):1223–9. doi:10.1016/j.jclinepi.2013.06.014.CrossRefPubMed

47.

Abbott A. The causal devolution. Sociol Methods Res. 1998;27(2):148–81. doi:10.1177/0049124198027002002.CrossRef

48.

Gough D, Oliver S, Thomas J. An introduction to systematic reviews. London: SAGE Publications; 2012.

49.

Creswell JW. Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks: SAGE Publications; 2013.

50.

Denzin NK, Lincoln YS. The Sage handbook of qualitative research. Thousand Oaks: SAGE Publications; 2005.

51.

Schwandt TA. The Sage dictionary of qualitative inquiry. Thousand Oaks: SAGE Publications; 2015.

52.

Glass GV. Primary, secondary, and meta-analysis of research. Educ Res. 1976;5:3–8. doi:10.3102/0013189X005010003.CrossRef

53.

Louis TA, Zelterman D. Bayesian approaches to research synthesis. In: Cooper H, Hedges LV, editors. The Handbook of Research Synthesis. New York: Russell Sage; 1994. p. 411–22.

54.

Larsson R. Case survey methodology: quantitative analysis of patterns across case studies. Acad Manage J. 1993;36(6):1515–46. doi:10.2307/256820.CrossRef

55.

Yin RK, Heald KA. Using the case survey method to analyze policy studies. Adm Sci Q. 1975;20(3):371–81. doi:10.2307/2391997.CrossRef

56.

Rihoux B, Ragin CC. Configurational comparative methods: qualitative comparative analysis (qca) and related techniques, vol. 51. Thousand Oaks: SAGE Publications; 2009.

57.

Droitcour J, Silberman G, Chelimsky E. Cross-design synthesis: a new form of meta-analysis for combining results from randomized clinical trials and medical-practice databases. Int J Technol Assess Health Care. 1993;9(03):440–9. doi:10.1017/S0266462300004694.CrossRefPubMed

58.

Sutton AJ, Higgins J. Recent developments in meta‐analysis. Stat Med. 2008;27(5):625–50. doi:10.1002/sim.2934.CrossRefPubMed

59.

Thompson SG, Higgins JP. How should meta-regression analyses be undertaken and interpreted? Stat Med. 2002;21(11):1559–73. doi:10.1002/sim.1187.CrossRefPubMed

60.

Sandelowski M, Barroso J, Voils CI. Using qualitative metasummary to synthesize qualitative and quantitative descriptive findings. Res Nurs Health. 2007;30(1):99–111. doi:10.1002/nur.20176.CrossRefPubMedPubMedCentral

61.

Light RJ, Smith PV. Accumulating evidence: procedures for resolving contradictions among different research studies. Harv Educ Rev. 1971;41(4):429–71. doi:10.17763/haer.41.4.437714870334w144.CrossRef

62.

Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J et al. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006;6(35). doi:10.1186/1471-2288-6-35.

63.

Booth A, Carroll C. How to build up the actionable knowledge base: the role of ‘best fit’ framework synthesis for studies of improvement in healthcare. BMJ quality & safety. 2015. doi:10.1136/bmjqs-2014-003642.

64.

Noblit GW, Hare RD. Meta-ethnography: synthesizing qualitative studies. Thousand Oaks: SAGE Publications; 1988.

65.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005;61:417–30. doi:10.1016/j.socscimed.2004.12.001.CrossRefPubMed

66.

Jensen LA, Allen MN. Meta-synthesis of qualitative findings. Qual Health Res. 1996;6(4):553–60. doi:10.1177/104973239600600407.CrossRef

67.

Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med. 2013;11(21). doi:10.1186/1741-7015-11-21.

68.

Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10 Suppl 1:21–34. doi:10.1258/1355819054308530.CrossRefPubMed

69.

Lucas PJ, Baird J, Arai L, Law C, Roberts HM. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Med Res Methodol. 2007;7:4. doi:10.1186/1471-2288-7-4.CrossRefPubMedPubMedCentral

70.

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45. doi:10.1186/1471-2288-8-45.CrossRefPubMedPubMedCentral

Title: Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence
Authors: Quan Nha Hong
Pierre Pluye
Mathieu Bujold
Maggy Wassef
Publication date: 01-12-2017
Publisher: BioMed Central
Published in: Systematic Reviews / Issue 1/2017
Electronic ISSN: 2046-4053
DOI: https://doi.org/10.1186/s13643-017-0454-2

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence

Abstract

Background

Methods

Results

Conclusions

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2017

Herceptin® (trastuzumab) in HER2-positive early breast cancer: protocol for a systematic review and cumulative network meta-analysis

External validation of type 2 diabetes computer simulation models: definitions, approaches, implications and room for improvement—a protocol for a systematic review

The efficacy of fall-risk-increasing drug (FRID) withdrawal for the prevention of falls and fall-related complications: protocol for a systematic review and meta-analysis

Management of frailty: a protocol of a network meta-analysis of randomized controlled trials

Rational and design of an individual participant data meta-analysis of spinal manipulative therapy for chronic low back pain—a protocol

Interventions to reduce dependency in bathing in community dwelling older adults: a systematic review