Top

Published in:

Open Access 01-12-2018 | Commentary

Partially systematic thoughts on the history of systematic reviews

Published in: Systematic Reviews | Issue 1/2018

Abstract

Six years after the launch of Systematic Reviews by Biomed Central, this article is part of the celebration of the journal. It contains personal reflections on the past, present and future of systematic reviews, using examples relevant to the role of systematic reviews in cataloguing and analysing research, assessing quality and planning new studies. The focus is on the most common of the various types of systematic review in health and social care, namely those assessing the effects of interventions.

Gould GM. The work of an association of medical librarians. J Med Libr Assoc. 1898;1:15–9.

Pearson K. Report on certain enteric fever inoculation statistics. Br Med J. 1904;3:1243–6.

Glass GV. Primary, secondary and meta-analysis of research. Educ Res. 1976;10:3–8.CrossRef

Gurevitch J, Koricheva J, Nakagawa S, Stewart G. Meta-analysis and the science of research synthesis. Nature. 2018;555:175–82.CrossRef

Clarke M. History of evidence synthesis to assess treatment effects: personal reflections on something that is very much alive. JLL bulletin: commentaries on the history of treatment evaluation. J R Soc Med. 2016;109:154–63.CrossRef

Chalmers I, Hedges LV, Cooper H. A brief history of research synthesis. Eval Health Prof. 2002;25:12–37.CrossRef

Starr M, Chalmers I, Clarke M, Oxman AD. The origins, evolution, and future of the Cochrane database of systematic reviews. Int J Technol Assess Health Care. 2009;25(suppl 1):182–95.CrossRef

Clarke M, Chalmers I. Reflections on the history of systematic reviews. BMJ Evid Based Med. 2018;23(4):121–2.CrossRef

Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and implications of rapid reviews. Implement Sci. 2010;5:56.CrossRef

10.

Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, Perrier L, Hutton B, Moher D, Straus SE. A scoping review of rapid review methods. BMC Med. 2015;13:224.CrossRef

11.

Elliott JH, Synnot A, Turner T, Simmonds M, Akl EA, McDonald S, Salanti G, Meerpohl J, MacLehose H, Hilton J, Tovey D, Shemilt I. Thomas J; living systematic review network. Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol. 2017;91:23–30.CrossRef

12.

Mulrow CD. Rationale for systematic reviews. BMJ. 1994;309:597–9.CrossRef

13.

Ghersi D, Pang T. From Mexico to Mali: four years in the history of clinical trial registration. J Evid Based Med. 2009;2(1):1–7.CrossRef

14.

Cochrane AL. 1931-1971: a critical review, with particular reference to the medical profession. In: Medicines for the year 2000. London: Office of Health Economics; 1979. p. 1–11.

15.

Chalmers I, Dickersin K, Chalmers TC. Getting to grips with Archie Cochrane’s agenda. BMJ. 1992;305:786–8.CrossRef

16.

Lind J. A treatise of the scurvy. In three parts. Containing an inquiry into the nature, causes and cure, of that disease. Together with a critical and chronological view of what has been published on the subject. Printed by Sands, Murray and Cochran for A Kincaid and A Donaldson: Edinburgh; 1753.

17.

Dickersin K, Manheimer E, Wieland S, Robinson KA, Lefebvre C, McDonald S, Central Development Group. Development of the Cochrane Collaboration’s central register of controlled clinical trials. Eval Health Prof. 2002;25:38–64.PubMed

18.

Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986;4(10):1529–41.CrossRef

19.

Powell-Smith A, Goldacre B. The TrialsTracker: automated ongoing monitoring of failure to share clinical trial results by all major companies and research institutions. F1000Research. 2629;2016:5.

20.

Chalmers I. Underreporting research is scientific misconduct. JAMA. 1990;263:1405–8.CrossRef

21.

Lee WL, Bausell RB, Berman BM. The growth of health-related meta-analyses published from 1980 to 2000. Eval Health Prof. 2001;24:327–35.CrossRef

22.

Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78.CrossRef

23.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7:e1000326.CrossRef

24.

Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, Catalá-López F, Li L, Reid EK, Sarkis-Onofre R, Moher D. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028.CrossRef

25.

Chalmers I, Enkin M, Keirse MJNC. Effective care in pregnancy and childbirth. Oxford: Oxford University Press; 1989.

26.

Allen C. A resource for those preparing for and responding to natural disasters, humanitarian crises, and major healthcare emergencies. J Evid Based Med. 2014;7:234–7.CrossRef

27.

Booth A, Clarke M, Dooley G, Ghersi D, Moher D, Petticrew M, Stewart L. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1:2.CrossRef

28.

Davies S. The importance of PROSPERO to the National Institute for Health Research. Syst Rev. 2012;1:5.CrossRef

29.

Page MJ, Shamseer L, Tricco AC. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev. 2018;7:32.CrossRef

30.

Ruano J, Gómez-García F, Gay-Mimbrera J, Aguilar-Luque M, Fernández-Rueda JL, Fernández-Chaichio J, Alcalde-Mellado P, Carmona-Fernandez PJ, Sanz-Cabanillas JL, Viguera-Guerra I, Franco-García F, Cárdenas-Aranzana M, Romero JLH, Gonzalez-Padilla M, Isla-Tejera B, Garcia-Nieto AV. Evaluating characteristics of PROSPERO records as predictors of eventual publication of non-Cochrane systematic reviews: a meta-epidemiological study protocol. Syst Rev. 2018;7:43.CrossRef

31.

Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7:39.CrossRef

32.

Adams CE, Polzmacher S, Wolff A. Systematic reviews: work that needs to be done and not to be done. J Evid Based Med. 2013;6:232–5.CrossRef

33.

Turner T, Green S, Tovey D, McDonald S, Soares-Weiser K, Pestridge C, Elliott J. On behalf of the project transform team and IKMD developers. Producing Cochrane systematic reviews—a qualitative study of current approaches and opportunities for innovation and improvement. Syst Rev. 2017;6:147.CrossRef

34.

Gates A, Johnson C, Hartling L. Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool. Syst Rev. 2018;7:45.CrossRef

35.

Rayleigh L. Address by the Rt. Hon. Lord Rayleigh. In: Report of the fifty-fourth meeting of the British Association for the Advancement of science; august and September; Montreal, Canada. London: John Murray; 1885.

36.

Smith ML, Glass GV. Meta-analysis of psychotherapy outcome studies. Am Psychol. 1977;32:752–60.CrossRef

37.

Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, Mantel N, McPherson K, Peto J, Smith PG. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Br J Cancer. 1977;35(1):1–39.CrossRef

38.

O'Rourke K. An historical perspective on meta-analysis: dealing quantitatively with varying study results. J R Soc Med. 2007;100:579–82.CrossRef

39.

Lewis S, Clarke M. Forest plots: trying to see the wood and the trees. BMJ. 2001;322:1479–80.CrossRef

40.

Lewis JA. Beta-blockade after myocardial infarction- a statistical view. Br J Clin Pharmacol. 1982;14:15S–21S.CrossRef

41.

Antiplatelet Trialists’ Collaboration. Secondary prevention of vascular disease by prolonged anti-platelet treatment. BMJ. 1988;296:320–31.CrossRef

42.

Clarke M, Stewart L, Pignon J-P, Bijnens L. Individual patient data meta-analyses in cancer. Br J Cancer. 1998;77:2036–44.CrossRef

43.

Simmonds M, Stewart G, Stewart LA. Decade of individual participant data meta-analyses: A review of current practice. Contemp Clin Trials. 2015;45(Pt A):76–83.CrossRef

44.

International Anticoagulant Review Group. Collaborative analysis of long-term anti-coagulant administration after acute myocardial infarction. Lancet. 1970;1:203–9.

45.

Varnai P, Rentel MC, Simmonds P, Sharp TA, Mostert B, de Jongh T. Assessing the research potential of access to clinical trial data. In: A report to the Wellcome Trust; 2014.

46.

Goldacre B, Lane S, Mahtani KR, Heneghan C, Onakpoya I, Bushfield I, Smeeth L. Pharmaceutical companies' policies on access to trial data, results, and methods: audit study. BMJ. 2017;358:j3334.CrossRef

47.

Mbuagbaw L, Rochwerg B, Jaeschke R, Heels-Andsell D, Alhazzani W, Thabane L, Guyatt GH. Approaches to interpreting and choosing the best treatments in network meta-analyses. Syst Rev. 2017;6:79.CrossRef

48.

Edwards SJ, Clarke MJ, Wordsworth S, Borrill J. Indirect comparisons of treatments based on systematic reviews of randomised controlled trials. Int J Clin Pract. 2009;63:841–54.CrossRef

49.

Lee AW. Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. J Clin Epidemiol. 2014;67:138–43.CrossRef

50.

Petropoulou M, Nikolakopoulou A, Veroniki AA, Rios P, Vafaei A, Zarin W, et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol. 2017;82:20–8.CrossRef

51.

Zarin W, Veroniki AA, Nincic V, Vafaei A, Reynen E, Motiwala SS, et al. Characteristics and knowledge synthesis approach for 456 network meta-analyses: a scoping review. BMC Med. 2017;15:3.CrossRef

52.

Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383:166–75.CrossRef

53.

Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA. Cochrane Bias methods group; Cochrane Statistical Methods Group The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.CrossRef

54.

Sterne JA, Hernán MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JP. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.CrossRef

55.

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ, GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–6.CrossRef

56.

Goldet G, Howick J. Understanding GRADE: an introduction. J Evid Based Med. 2013;6:50–4.CrossRef

57.

Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gülmezoglu M, Noyes J, Booth A, Garside R, Rashidian A. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med. 2015;12(10):e1001895.CrossRef

58.

Oxman AD. Checklists for review articles. BMJ. 1994;309:648–51.CrossRef

59.

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.CrossRef

60.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.CrossRef

61.

Hoaglin DC, Hawkins N, Jansen JP, Scott DA, Itzler R, Cappelleri JC, Boersma C, Thompson D, Larholt KM, Diaz M, Barrett A. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR task force on indirect treatment comparisons good research practices: part 2. Value Health. 2011;14(4):429–37.CrossRef

62.

Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999;354:1896–900.CrossRef

63.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100.CrossRef

64.

Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, Tierney JF, PRISMA-IPD Development Group. Preferred reporting items for systematic review and meta-analyses of individual participant data: the PRISMA-IPD statement. JAMA. 2015;313:1657–65.CrossRef

65.

Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, Ioannidis JP, Straus S, Thorlund K, Jansen JP, Mulrow C, Catalá-López F, Gøtzsche PC, Dickersin K, Boutron I, Altman DG, Moher D. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162:777–84.CrossRef

66.

Pussegoda K, Turner L, Garritty C, Mayhew A, Skidmore B, Stevens A, Boutron I, Sarkis-Onofre R, Bjerre LM, Hróbjartsson A, Altman DG, Moher D. Systematic review adherence to methodological or reporting quality. Syst Rev. 2017;6:131.CrossRef

67.

Ioannidis JPA. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94:485–514.CrossRef

68.

Shaikh W, Vayda E, Feldman W. A systematic review of the literature on evaluative studies on tonsillectomy and adenoidectomy. Pediatrics. 1976;57:401–7.PubMed

69.

Clarke L, Clarke M, Clarke T. How useful are Cochrane reviews in identifying research needs? J Health Serv Res Policy. 2007;12:101–3.CrossRef

70.

Clarke M. Doing new research? Don’t forget the old: nobody should do a trial without reviewing what is known. PLoS Med. 2004;1:100–2.CrossRef

71.

Clarke M, Hopewell S. Many reports of randomised trials still don’t begin or end with a systematic review of the relevant evidence. J Bahrain Med Soc. 2013;24:145–8.

72.

Robinson KA, Goodman SN. A systematic examination of the citation of prior research in reports of randomized, controlled trials. Ann Intern Med. 2011;154:50–5.CrossRef

73.

Treweek S, Altman DG, Bower P, Campbell M, Chalmers I, Cotton S, Craig P, Crosby D, Davidson P, Devane D, Duley L, Dunn J, Elbourne D, Farrell B, Gamble C, Gillies K, Hood K, Lang T, Littleford R, Loudon K, McDonald A, McPherson G, Nelson A, Norrie J, Ramsay C, Sandercock P, Shanahan DR, Summerskill W, Sydes M, Williamson P, Clarke M. Making randomised trials more efficient: report of the first meeting to discuss the trial forge platform. Trials. 2015;16:261.CrossRef

74.

McKenzie JE, Clarke MJ, Chandler J. Why do we need evidence-based methods in Cochrane? Cochrane Database Syst Rev. 2015;7:ED000102.

75.

Antes G, Clarke M. Knowledge as a key resource for health challenges. Lancet. 2012;379:195–6.CrossRef

Title: Partially systematic thoughts on the history of systematic reviews
Publication date: 01-12-2018
Published in: Systematic Reviews / Issue 1/2018
Electronic ISSN: 2046-4053
DOI: https://doi.org/10.1186/s13643-018-0833-3

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Partially systematic thoughts on the history of systematic reviews

Abstract

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2018

A review of recent publication trends from top publishing countries

Prognostic value of microRNAs in head and neck cancers: a systematic review and meta-analysis protocol

Protocol for a systematic review of venous coupler devices versus hand-sewn anastomosis for microsurgical free flap reconstruction

An introduction to overviews of reviews: planning a relevant research question and objective for an overview

Experiences of aging in place in the United States: protocol for a systematic review and meta-ethnography of qualitative studies

Diagnostic accuracy of molecular methods for detecting markers of antimalarial drug resistance in clinical samples of Plasmodium falciparum: protocol for an update to a systematic review and meta-analysis