Skip to main content
Top
Published in: Systematic Reviews 1/2015

Open Access 01-12-2015 | Commentary

Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’

Author: Mark Petticrew

Published in: Systematic Reviews | Issue 1/2015

Login to get access

Abstract

Systematic review methods are developing rapidly, and most researchers would recognise their key methodological aspects, such as a closely focussed question, a comprehensive search, and a focus on synthesising ‘stronger’ rather than ‘weaker’ evidence. However, it may be helpful to question some of these underlying principles, because while they work well for simpler review questions, they may result in overly narrow approaches to more complex questions and interventions. This commentary discusses some core principles of systematic reviews, and how they may require further rethinking, particularly as reviewers turn their attention to increasingly complex issues, where a Bayesian perspective on evidence synthesis, which would aim to assemble evidence - of different types, if necessary - in order to inform decisions’, may be more productive than the ‘traditional’ systematic review model. Among areas identified for future research are the examination of publication bias in qualitative research; research on the efficiency and potential biases of comprehensive searches in different disciplines; and the use of Bayesian methods in evidence synthesis. The incorporation of a systems perspective into systematic reviews is also an area which needs rapid development.
Literature
2.
go back to reference Chandler J, Hopewell S. Cochrane methods–twenty years experience in developing systematic review methods. Syst Rev. 2013;2:76. doi:10.1186/2046-4053-2-76.CrossRefPubMedPubMedCentral Chandler J, Hopewell S. Cochrane methods–twenty years experience in developing systematic review methods. Syst Rev. 2013;2:76. doi:10.1186/2046-4053-2-76.CrossRefPubMedPubMedCentral
3.
go back to reference Schünemann H, Tugwell P, Reeves B, Akl EA, Santesso N, Spencer FA, et al. Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Res Syn Meth. 2013;4:49–62.CrossRef Schünemann H, Tugwell P, Reeves B, Akl EA, Santesso N, Spencer FA, et al. Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Res Syn Meth. 2013;4:49–62.CrossRef
4.
go back to reference Petticrew M, Roberts H. Systematic reviews in the social sciences. A practical guide Oxford: Blackwell; 2006.CrossRef Petticrew M, Roberts H. Systematic reviews in the social sciences. A practical guide Oxford: Blackwell; 2006.CrossRef
5.
go back to reference Whelan J, Love P, Pettman T, Doyle J, Booth S, Smith E, et al. Cochrane update: predicting sustainability of intervention effects in public health evidence: identifying key elements to provide guidance. J Public Health (Oxf). 2014;36(2):347–51.CrossRef Whelan J, Love P, Pettman T, Doyle J, Booth S, Smith E, et al. Cochrane update: predicting sustainability of intervention effects in public health evidence: identifying key elements to provide guidance. J Public Health (Oxf). 2014;36(2):347–51.CrossRef
6.
go back to reference Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, et al. Integrating qualitative research with trials in systematic reviews. Br Med J. 2004;328:1010–2.CrossRef Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, et al. Integrating qualitative research with trials in systematic reviews. Br Med J. 2004;328:1010–2.CrossRef
8.
go back to reference Campbell R, Britten N, Pound P, Donovan J, Morgan M, Pill P, et al. Using meta-ethnography to synthesise qualitative research, Moving beyond effectiveness in evidence synthesis. London: National Institute for Health and Clinical Excellence; 2006. p. 75–82. Campbell R, Britten N, Pound P, Donovan J, Morgan M, Pill P, et al. Using meta-ethnography to synthesise qualitative research, Moving beyond effectiveness in evidence synthesis. London: National Institute for Health and Clinical Excellence; 2006. p. 75–82.
10.
go back to reference Greenhalgh T, Wong G, Westhorp G, Pawson R. Protocol - realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol. 2011;11:115.CrossRefPubMedPubMedCentral Greenhalgh T, Wong G, Westhorp G, Pawson R. Protocol - realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol. 2011;11:115.CrossRefPubMedPubMedCentral
12.
go back to reference Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med. 2013;11(21) doi:10.1186/1741-7015-11-21. Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med. 2013;11(21) doi:10.1186/1741-7015-11-21.
13.
go back to reference Hannes K, Booth A, Harris J, Noyes J. Celebrating methodological challenges and changes: reflecting on the emergence and importance of the role of qualitative evidence in Cochrane reviews. Syst Rev. 2013;2:84.CrossRefPubMedPubMedCentral Hannes K, Booth A, Harris J, Noyes J. Celebrating methodological challenges and changes: reflecting on the emergence and importance of the role of qualitative evidence in Cochrane reviews. Syst Rev. 2013;2:84.CrossRefPubMedPubMedCentral
14.
go back to reference Sackett D, Haynes R, Guyatt G, Tugwell P. Clinical epidemiology. Boston: Little, Brown: A basic science for clinical medicine; 1991. Sackett D, Haynes R, Guyatt G, Tugwell P. Clinical epidemiology. Boston: Little, Brown: A basic science for clinical medicine; 1991.
15.
go back to reference Mulrow CD. Rationale for systematic reviews. Br Med J. 1994;309(6954):597–9.CrossRef Mulrow CD. Rationale for systematic reviews. Br Med J. 1994;309(6954):597–9.CrossRef
16.
go back to reference Gough D, Oliver S, Thomas J. An introduction to systematic reviews. London: SAGE Publications Ltd; 2012. Gough D, Oliver S, Thomas J. An introduction to systematic reviews. London: SAGE Publications Ltd; 2012.
17.
go back to reference Sandelowski M, Voils C, Leeman J, Crandell JL. Mapping the mixed methods–mixed research synthesis terrain. J Mix Methods Res. 2012;6(4):317–31.CrossRefPubMed Sandelowski M, Voils C, Leeman J, Crandell JL. Mapping the mixed methods–mixed research synthesis terrain. J Mix Methods Res. 2012;6(4):317–31.CrossRefPubMed
18.
go back to reference Higgins J, Green S, editors. Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]: The Cochrane Collaboration. 2011. Available from www.cochrane-handbook.org., 2011. Higgins J, Green S, editors. Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]: The Cochrane Collaboration. 2011. Available from www.​cochrane-handbook.​org., 2011.
19.
go back to reference Moher D. The problem of duplicate systematic reviews. Br Med J. 2013;347:f5040.CrossRef Moher D. The problem of duplicate systematic reviews. Br Med J. 2013;347:f5040.CrossRef
20.
go back to reference Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. PROSPERO at one year: an evaluation of its utility. Syst Rev. 2013;15(2):4.CrossRef Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, et al. PROSPERO at one year: an evaluation of its utility. Syst Rev. 2013;15(2):4.CrossRef
21.
go back to reference Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43(3–4):267–76.CrossRefPubMed Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43(3–4):267–76.CrossRefPubMed
22.
go back to reference Shea B, Grimshaw J, Wells G, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.CrossRefPubMedPubMedCentral Shea B, Grimshaw J, Wells G, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.CrossRefPubMedPubMedCentral
23.
go back to reference Lorenc T, Pearson R, Jamal F, Cooper C, Garside R. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Res Synth Methods. 2012;3(1):1–10.CrossRefPubMed Lorenc T, Pearson R, Jamal F, Cooper C, Garside R. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Res Synth Methods. 2012;3(1):1–10.CrossRefPubMed
24.
go back to reference Claxton K, Sculpher M. Using value of information analysis to prioritise health research - some lessons from recent UK experience. Pharmacoeconomics. 2006;24:1055–68.CrossRefPubMed Claxton K, Sculpher M. Using value of information analysis to prioritise health research - some lessons from recent UK experience. Pharmacoeconomics. 2006;24:1055–68.CrossRefPubMed
25.
go back to reference Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431–5.CrossRefPubMed Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431–5.CrossRefPubMed
26.
go back to reference Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Health Technol Assess. 2003;7(1):1–76.PubMed Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Health Technol Assess. 2003;7(1):1–76.PubMed
27.
go back to reference Petticrew M. Plural evidence in public health evaluation: epistemological challenges to evidence production and use. Evid Policy. 2013;9(1):87–95.CrossRef Petticrew M. Plural evidence in public health evaluation: epistemological challenges to evidence production and use. Evid Policy. 2013;9(1):87–95.CrossRef
28.
go back to reference Ogilvie D, Egan M, Hamilton V, Petticrew M. Promoting walking and cycling as an alternative to using cars: what works? A systematic review. Br Med J. 2004;329(7469):763.CrossRef Ogilvie D, Egan M, Hamilton V, Petticrew M. Promoting walking and cycling as an alternative to using cars: what works? A systematic review. Br Med J. 2004;329(7469):763.CrossRef
29.
go back to reference Ogilvie D, Egan M, Hamilton V, Petticrew M. Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go? J Epidemiol Community Health. 2005;59:886–92.CrossRefPubMedPubMedCentral Ogilvie D, Egan M, Hamilton V, Petticrew M. Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go? J Epidemiol Community Health. 2005;59:886–92.CrossRefPubMedPubMedCentral
30.
go back to reference Popay J. Plenary presentation. Sao Paolo: Cochrane Colloquium; 2007. Popay J. Plenary presentation. Sao Paolo: Cochrane Colloquium; 2007.
32.
go back to reference Gibson M, Thomson H, Kearns A, Petticrew M. Understanding the psychosocial impacts of housing type: qualitative evidence from a housing and regeneration intervention. Hous Stud. 2011;26(4):555–73.CrossRef Gibson M, Thomson H, Kearns A, Petticrew M. Understanding the psychosocial impacts of housing type: qualitative evidence from a housing and regeneration intervention. Hous Stud. 2011;26(4):555–73.CrossRef
33.
go back to reference Thomson H, Thomas S, Sellström E, Petticrew M. Housing improvements for health and associated socio-economic outcomes: a systematic review. Campbell Syst Rev. 2013. p. 2. doi: 104073/csr20132. Thomson H, Thomas S, Sellström E, Petticrew M. Housing improvements for health and associated socio-economic outcomes: a systematic review. Campbell Syst Rev. 2013. p. 2. doi: 104073/csr20132.
34.
go back to reference Roberts K, Dixon-Woods M, Fitzpatrick R, Abrams KR, Jones DR. Factors affecting uptake of childhood immunisation: a Bayesian synthesis of qualitative and quantitative evidence. Lancet. 2002;360:1596–9.CrossRefPubMed Roberts K, Dixon-Woods M, Fitzpatrick R, Abrams KR, Jones DR. Factors affecting uptake of childhood immunisation: a Bayesian synthesis of qualitative and quantitative evidence. Lancet. 2002;360:1596–9.CrossRefPubMed
35.
go back to reference Threlfall AG, Meah S, Fischer AJ, Cookson R, Rutter H, Kelly MP. The appraisal of public health interventions: the use of theory. J Public Health. 2015;37(1):166–71.CrossRef Threlfall AG, Meah S, Fischer AJ, Cookson R, Rutter H, Kelly MP. The appraisal of public health interventions: the use of theory. J Public Health. 2015;37(1):166–71.CrossRef
Metadata
Title
Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’
Author
Mark Petticrew
Publication date
01-12-2015
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2015
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/s13643-015-0027-1

Other articles of this Issue 1/2015

Systematic Reviews 1/2015 Go to the issue