Top

Published in:

Open Access 01-12-2019 | Commentary

Clarifying differences between reviews within evidence ecosystems

Authors: David Gough, James Thomas, Sandy Oliver

Published in: Systematic Reviews | Issue 1/2019

Abstract

This paper builds on a 2012 paper by the same authors which argued that the types and brands of systematic review do not sufficiently differentiate between the many dimensions of different review questions and review methods (Gough et al., Syst Rev 1:28, 2012). The current paper extends this argument by considering the dynamic contexts, or ‘evidence ecosystems’, within which reviews are undertaken; the fact that these ecosystems are constantly changing; and the relevance of this broader context for understanding ‘dimensions of difference’ in the unfolding development and refinement of review methods.

The concept of an evidence ecosystem is used to consider particular issues within the three key dimensions of difference outlined in the 2012 paper of (1) review aims and approach, (2) structure and components of reviews, and (3) breadth, depth, and ‘work done’ by reviews.

Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev. 2012;1:28.PubMedPubMedCentralCrossRef

Gough D, Oliver S, Thomas J. An introduction to systematic reviews: 2^nd Edition. London: Sage; 2017.

Best A, Holmes B. Systems thinking, knowledge and action: towards better models and methods’, Evidence & Policy: A Journal of Research. Debate and Pract. 2010;6(2):145–59.CrossRef

Nowotny H. The potential of transdisciplinarity. Available at: http://www.helga-nowotny.eu/downloads/helga_nowotny_b59.pdf

Gough D, Maidment C, Sharples J (2018). UK What Works Centres: aims, methods and contexts. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London. ISBN: 978-1-911605-03-4

Parkhurst J. The politics of evidence. From evidence-based policy to the good governance. London: Routledge; 2016.

Langer L, Tripney J, Gough D. The science of using science: researching the use of sesearch evidence in decision-making. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London; 2016. p. 2016.

Oliver S, Dickson K. Policy-relevant systematic reviews to strengthen health systems: models and mechanisms to support their production. Evidence and Policy. 2016;12(2):235–59.CrossRef

Oliver S, Bangpan M, Dickson K. Producing policy relevant systematic reviews: navigating the policy-research interface. Evidence and Policy; 2017.

10.

Weiss C. The many meanings of research utilization. Public Adm Rev. 1979;39(5):426–31.CrossRef

11.

Gough DA. Evidence Standards: A dimensions of difference framework for appraising justifiable evidence claims London: EPPI-Centre, UCL; 2016. https://eppi.ioe.ac.uk/CMS/Portals/0/PDF%20reviews%20and%20summaries/Evidence%20standards%20for%20justifiable%20evidence%20claims%20DG%20June%202016%20final.pdf

12.

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence Lancet. 2009; 4; 374(9683) 86-89.

13.

Gough D, Kenny C, Vigurs C, Stansfield C, Rosen R, Taylor T. Social values related to the development of health and care guidance: literature review for NICE by its Research Support Unit. Report of the Research Support Unit for the National Institute for Health and Care Excellence. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2014.

14.

Welch V, Petticrew M, Petkovic J, Moher D, Waters E, White H, et al. Extending the PRISMA statement to equity-focused systematic reviews (PRISMA-E 2012): explanation and elaboration. J Clin Epidemiol. 2016; 70:68-89. doi: https://doi.org/10.1016/j.jclinepi.2015.09.001. Epub 2015 Sep 5.PubMedCrossRef

15.

Oliver S, Hollingworth K, Briner B, Swann C, Hinds K, Roche C. Effective and efficient committee work: a systematic overview of multidisciplinary literatures. Evid Base. 2018, 2018;(2):1–21. https://doi.org/10.21307/eb-2018-002.CrossRef

16.

Oliver S, Roche C, Stewart R, Bangpan M, Dickson K, Pells K, Cartwright N, Hargreaves J, Gough D, Stakeholder engagement for development impact evaluation and evidence synthesis CEDIL Inception Paper 3: London; 2018.

17.

Hume, D. An enquiry concerning human understanding, Oxford: Oxford University Press. ed and intro P. Millican. First published 1748; 2007.

18.

Kneale D, Thomas J, Harris K. Developing and optimising the use of logic models in systematic reviews: exploring practice and good practice in the use of Programme Theory in reviews. PloS One. 2015;b10:e0142187.CrossRef

19.

Rehfuess EA, Booth A, Brereton L, Burns J, Gerhardus A, Mozygemba K, Oortwijn W, Pfadenhauer LM, Tummers M, van der Wilt GJ, Rohwer A. Towards a taxonomy of logic models in systematic reviews and health technology assessments: a priori, staged, and iterative approaches. Res Synth Methods. 2018;9:13–24.PubMedCrossRef

20.

Kneale D, Thomas J, Bangpan M, Waddington H. Conceptualising causal pathways in systematic reviews of international development interventions through adopting a causal chain analysis approach. J Dev Eff. 2018:422–37.CrossRef

21.

Thomas J, O’Mara-Eves A, Brunton G. Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example. Syst Rev. 2014;3:67.PubMedPubMedCentralCrossRef

22.

Petticrew M, Knai C, Thomas J, Rehfuess E, Noyes J, Gerhardus A, Grimshaw J, Rutter H, McGill E. Implications of a complex systems perspective perspective for systematic reviews and guideline development in health decision-making. BMJ Global Health. 2019;4(Suppl1):e000899.PubMedPubMedCentralCrossRef

23.

Higgins JPT, López-López JA, Becker BJ, Davies SR, Dawson S, Grimshaw JM, McGuinness LA, Moore THM, Rehfuess E, Thomas J, Caldwell DM. Synthesizing quantitative evidence in systematic reviews of complex health interventions. BMJ Global Health. 2019;4(Suppl1):e000858.PubMedPubMedCentralCrossRef

24.

Stewart LA, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof. 2002 Mar;25(1):76–97.PubMedCrossRef

25.

Chowkwanyun M, Bayer R, Galea SJ. “Precision” public health—between novelty and hype. N Eng J Med. 2018;379:1398–400.CrossRef

26.

Kneale D, Thomas J, O’Mara-Eves A, Wiggins R . How can additional secondary data analysis of observational data enhance the generalisability of meta-analytic evidence for local public health decision-making? Res Synth Methods. 2018 [online first 21 August]

27.

Moher D, Liberati A, Tetzlaff J, Altman DG. The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed1000097.PubMedPubMedCentralCrossRef

28.

Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med. 2013; Jan 29;11:21.PubMedPubMedCentralCrossRef

29.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.PubMedPubMedCentralCrossRef

30.

Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. January 2016;2016(69):225–34.CrossRef

31.

Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, et al. GRADE Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336:1106.PubMedPubMedCentralCrossRef

32.

Lewin S, Booth A, Glenton C, Munthe-Kaas H, Rashidian A, Wainwright M, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. 2018;13(Suppl 1):2.PubMedPubMedCentralCrossRef

33.

Gough D, White H. Evidence standards and evidence claims in web based research portals. London: Centre for Homelessness Impact. 2018. ISBN: 978-1-9995928-3-7

34.

Thomas J, Noel-Storr A, McDonald S (2019) Evidence surveillance: how automation technologies and working processes can enable us to keep up to date with new research. In: Craven J, Levay P. Systematic Searching: Practical ideas for improving results. Facet Publishing. 2019.

35.

Chalmers I. Underreporting research is scientific misconduct. JAMA. 1990;263:1405–8.PubMedCrossRef

36.

Michie S, Thomas J, Johnston M, Aonghusa PM, Shawe-Taylor J, Kelly MP, et al.. The Human Behaviour-Change Project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation. Michie et al. Implementation Science. 2017; 12:121

37.

Elliott J, Turner TJ, Clavisi OM, Thomas J, Higgins JPT, Mavergames C, Gruen RL. PLoS Med. 2014;11:2.CrossRef

38.

Thomas J, Noel-Storr A, Marshall I, Wallace B, McDonald S, Mavergames C et al. Living systematic reviews: 2. Combining human and machine effort. Journal of Clinical Epidemiology; 2017.

39.

Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016:353.

40.

Rehfuess EA, Stratil JM, Scheel IB, et al. The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective. BMJ Global Health. 2019;4:e000844.PubMedPubMedCentralCrossRef

41.

Pawson R. Evidence-based policy: the promise of “Realist Synthesis” ’. Evaluation. 2002;8(3):340–58.CrossRef

42.

Elliott J, Synnot A, Turner T, Simmonds M, Akl E, McDonald S, Salanti G, Thomas J, Meerpohl J, MacLehose H, Hilton J, Shemilt I, Tovey D on behalf of the Living Systematic Review Network. Living systematic review: 1. Introduction-the why, what, when, and how. Journal of Clinical Epidemiology. 2017;91:23-30.PubMedCrossRef

43.

Chaimani A, Caldwell DM, Li T, Higgins JPT, Salanti G. Chapter 11: Undertaking network meta-analyses. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane handbook for systematic reviews of interventions. 2nd Edition. Chichester (UK): John Wiley & Sons, 2019.

44.

Caldwell DM. An overview of conducting systematic reviews with network meta-analysis. Syst Rev [Internet]. 2014;3(1):109.

45.

Gough D, Thomas J. Commonality and diversity in reviews. In: Gough D, Oliver S, Thomas J, editors. An introduction to systematic reviews: 2nd Edition. London: Sage; 2017.

46.

Peersman G. (1996) A descriptive mapping of health promotion studies in young people, EPPI Research Report. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 1996.

47.

Gough DA, Kiwan D, Sutcliffe S, Simpson D, Houghton N. A systematic map and synthesis review of the effectiveness of personal development planning for improving student learning. London: EPPI-Centre, Social Science Research Unit. 2003. ISBN 0954203666

48.

Caird J, Sutcliffe K, Kwan I, Dickson K, Thomas J. Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? Evid Policy, 2015; 11; 1. 81-97CrossRef

Title: Clarifying differences between reviews within evidence ecosystems
Authors: David Gough
James Thomas
Sandy Oliver
Publication date: 01-12-2019
Publisher: BioMed Central
Published in: Systematic Reviews / Issue 1/2019
Electronic ISSN: 2046-4053
DOI: https://doi.org/10.1186/s13643-019-1089-2

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Clarifying differences between reviews within evidence ecosystems

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2019

Mapping the evidence of hepatoprotective properties of Moringa oleifera from sub-Saharan African countries: a systematic review protocol

Interventions to promote patients and families’ involvement in adult intensive care settings: a protocol for a mixed-method systematic review

The efficacy of cognitive interventions on the performance of instrumental activities of daily living in individuals with mild cognitive impairment or mild dementia: protocol for a systematic review and meta-analysis

Communication around HPV vaccination for adolescents in low- and middle-income countries: a systematic scoping overview of systematic reviews

The effect of acute stress on salivary markers of inflammation: a systematic review protocol

Usability and acceptability of four systematic review automation software packages: a mixed method design