Skip to main content
Top
Published in: International Orthopaedics 3/2012

01-03-2012 | Review Article

Steps in the undertaking of a systematic review in orthopaedic surgery

Authors: Dario Sambunjak, Miljenko Franić

Published in: International Orthopaedics | Issue 3/2012

Login to get access

Abstract

In the last decades of the twentieth century it became obvious that modern medical care is replete with data and information, but in need of reliable evidence. This has led to an increased effort to systematically synthesise medical research and make it more useful for practitioners. Systematic reviews use an approach to research synthesis that minimises the risk of misinterpreting a body of evidence due to incomprehensive search or subjective opinion. Carrying out a systematic review is a rigorous procedure which corresponds to standard methodological steps in primary research studies. It involves posing a well-defined question, developing a robust search strategy, screening for relevant primary studies, critical appraisal of included studies, data extraction and processing, analysis and interpretation of results. In some, but not all systematic reviews it is appropriate to conduct a meta-analysis, which is a statistical procedure that integrates the results of several independent studies. Results of meta-analysis are graphically presented in forest plots, with pooled point estimate and its confidence interval represented as a rhombus, usually called a “diamond”. Methodological quality of systematic reviews should not be judged by the quality of primary studies included, but by a distinct set of criteria specified in assessment tools such as AMSTAR. Systematic reviews and meta-analyses should be reported according to the PRISMA checklist. A major contribution to the development of methodological standards has been given by The Cochrane Collaboration, whose Handbook of Systematic Reviews of Interventions is the primary reference for all authors and referees of systematic reviews in health care.
Literature
1.
go back to reference Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS (1996) Evidence based medicine: what it is and what it isn’t. BMJ 312:71–72PubMedCrossRef Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS (1996) Evidence based medicine: what it is and what it isn’t. BMJ 312:71–72PubMedCrossRef
2.
go back to reference Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC (1992) A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 268:240–248PubMedCrossRef Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC (1992) A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 268:240–248PubMedCrossRef
3.
go back to reference Ernst E, Pittler MH (2001) Assessment of therapeutic safety in systematic reviews: literature review. BMJ 323:546PubMedCrossRef Ernst E, Pittler MH (2001) Assessment of therapeutic safety in systematic reviews: literature review. BMJ 323:546PubMedCrossRef
5.
go back to reference Wale J, Colombo C, Belizan M, Nadel J (2010) International health consumers in the Cochrane Collaboration: fifteen years on. J Ambul Care Manage 33:182–189PubMed Wale J, Colombo C, Belizan M, Nadel J (2010) International health consumers in the Cochrane Collaboration: fifteen years on. J Ambul Care Manage 33:182–189PubMed
6.
go back to reference Wale JL, Belizán M, Nadel J, Jeffrey C, Vij SL (2011) The Cochrane Library review titles that are important to users of health care, a Cochrane Consumer Network project. Health Expect [Epub ahead of print]. doi: 10.1111/j.1369-7625.2011.00723.x Wale JL, Belizán M, Nadel J, Jeffrey C, Vij SL (2011) The Cochrane Library review titles that are important to users of health care, a Cochrane Consumer Network project. Health Expect [Epub ahead of print]. doi: 10.​1111/​j.​1369-7625.​2011.​00723.​x
7.
go back to reference Straus SE, Richardson WS, Glasziou P, Haynes RB (2005) Evidence-based medicine: how to practice and teach EBM, 3rd edn. Churchill Livingstone, Edinburgh Straus SE, Richardson WS, Glasziou P, Haynes RB (2005) Evidence-based medicine: how to practice and teach EBM, 3rd edn. Churchill Livingstone, Edinburgh
9.
go back to reference Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J et al (2011) GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 64:401–406. doi:dx.doi.org PubMedCrossRef Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J et al (2011) GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 64:401–406. doi:dx.​doi.​org PubMedCrossRef
10.
go back to reference Reeves BC, Deeks JJ, Higgins JPT, Wells GA, on behalf of the Cochrane Non-Randomised Studies Methods Group (2011) Chapter 13: Including non-randomized studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011 Reeves BC, Deeks JJ, Higgins JPT, Wells GA, on behalf of the Cochrane Non-Randomised Studies Methods Group (2011) Chapter 13: Including non-randomized studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.​cochrane-handbook.​org. Accessed 14 Nov 2011
11.
go back to reference Clifton R, Haleem S, McKee A, Parker MJ (2008) Closed suction surgical wound drainage after hip fracture surgery: a systematic review and meta-analysis of randomised controlled trials. Int Orthop 32:723–727PubMedCrossRef Clifton R, Haleem S, McKee A, Parker MJ (2008) Closed suction surgical wound drainage after hip fracture surgery: a systematic review and meta-analysis of randomised controlled trials. Int Orthop 32:723–727PubMedCrossRef
12.
go back to reference Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C (2000) Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials 21:476–487PubMedCrossRef Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C (2000) Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials 21:476–487PubMedCrossRef
13.
go back to reference Lefebvre C, Manheimer E, Glanville J, on behalf of the Cochrane Information Retrieval Methods Group (2011) Chapter 6: Searching for studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011 Lefebvre C, Manheimer E, Glanville J, on behalf of the Cochrane Information Retrieval Methods Group (2011) Chapter 6: Searching for studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.​cochrane-handbook.​org. Accessed 14 Nov 2011
14.
go back to reference Mead TL, Richards DT (1995) Librarian participation in meta-analysis projects. Bull Med Libr Assoc 83:461–464PubMed Mead TL, Richards DT (1995) Librarian participation in meta-analysis projects. Bull Med Libr Assoc 83:461–464PubMed
15.
go back to reference Bates MJ (1989) The design of browsing and berrypicking techniques for the online search interface. Online Review 13:407–424CrossRef Bates MJ (1989) The design of browsing and berrypicking techniques for the online search interface. Online Review 13:407–424CrossRef
16.
go back to reference Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, Godlee F et al (2007) Clinical trial registration: looking back and moving ahead. JAMA 298:93–94PubMedCrossRef Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, Godlee F et al (2007) Clinical trial registration: looking back and moving ahead. JAMA 298:93–94PubMedCrossRef
17.
go back to reference Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K (2009) Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 1:MR000006 Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K (2009) Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 1:MR000006
18.
go back to reference Ibrahim T, Tleyjeh IM, Gabbar O (2008) Surgical versus non-surgical treatment of chronic low back pain: a meta-analysis of randomised trials. Int Orthop 32:107–113PubMedCrossRef Ibrahim T, Tleyjeh IM, Gabbar O (2008) Surgical versus non-surgical treatment of chronic low back pain: a meta-analysis of randomised trials. Int Orthop 32:107–113PubMedCrossRef
19.
go back to reference Bero LA, Grilli R, Grimshaw J, Harvey E, Oxman AD (1997) External refereeing of protocols for systematic reviews [abstract]. Prague, Czech Republic, Third International Congress on Biomedical Peer Review and Global Communications Bero LA, Grilli R, Grimshaw J, Harvey E, Oxman AD (1997) External refereeing of protocols for systematic reviews [abstract]. Prague, Czech Republic, Third International Congress on Biomedical Peer Review and Global Communications
20.
go back to reference Silagy CA, Middleton P, Hopewell S (2002) Publishing protocols of systematic reviews. Comparing what was done to what was planned. JAMA 287:2831–2834PubMedCrossRef Silagy CA, Middleton P, Hopewell S (2002) Publishing protocols of systematic reviews. Comparing what was done to what was planned. JAMA 287:2831–2834PubMedCrossRef
21.
go back to reference Kirkham JJ, Altman DG, Williamson PR (2010) Bias due to changes in specified outcomes during the systematic review process. PLoS One 5(3):e9810PubMedCrossRef Kirkham JJ, Altman DG, Williamson PR (2010) Bias due to changes in specified outcomes during the systematic review process. PLoS One 5(3):e9810PubMedCrossRef
22.
go back to reference Glasziou P, Sanders S, Pirozzo S, Doust J, Pietrzak E (2002) Abstract screening—the value of two reviewers [abstract]. Pushing the Boundaries, Oxford, United Kingdom, Fourth Symposium on Systematic Reviews Glasziou P, Sanders S, Pirozzo S, Doust J, Pietrzak E (2002) Abstract screening—the value of two reviewers [abstract]. Pushing the Boundaries, Oxford, United Kingdom, Fourth Symposium on Systematic Reviews
23.
go back to reference Stavlas P, Roberts CS, Xypnitos FN, Giannoudis PV (2010) The role of reduction and internal fixation of Lisfranc fracture-dislocations: a systematic review of the literature. Int Orthop 34:1083–1091PubMedCrossRef Stavlas P, Roberts CS, Xypnitos FN, Giannoudis PV (2010) The role of reduction and internal fixation of Lisfranc fracture-dislocations: a systematic review of the literature. Int Orthop 34:1083–1091PubMedCrossRef
24.
go back to reference Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP (2006) Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol 59:697–703PubMedCrossRef Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP (2006) Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol 59:697–703PubMedCrossRef
25.
go back to reference Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C et al (2006) Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust 185:263–267PubMed Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C et al (2006) Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust 185:263–267PubMed
26.
go back to reference Higgins JPT, Deeks JJ (2011) Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration Higgins JPT, Deeks JJ (2011) Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration
27.
go back to reference Cheng T, Feng JG, Liu T, Zhang XL (2009) Minimally invasive total hip arthroplasty: a systematic review. Int Orthop 33:1473–1481PubMedCrossRef Cheng T, Feng JG, Liu T, Zhang XL (2009) Minimally invasive total hip arthroplasty: a systematic review. Int Orthop 33:1473–1481PubMedCrossRef
28.
go back to reference Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S (1995) Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 16:62–73PubMedCrossRef Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S (1995) Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 16:62–73PubMedCrossRef
29.
go back to reference Higgins JPT, Altman DG, Sterne JAC on behalf of the Cochrane Statistical Methods Group and the Cochrane Bias Methods Group (2011) Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011 Higgins JPT, Altman DG, Sterne JAC on behalf of the Cochrane Statistical Methods Group and the Cochrane Bias Methods Group (2011) Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.​cochrane-handbook.​org. Accessed 14 Nov 2011
30.
go back to reference Xue D, Zheng Q, Li H, Qian S, Zhang B, Pan Z (2011) Selective COX-2 inhibitor versus nonselective COX-1 and COX-2 inhibitor in the prevention of heterotopic ossification after total hip arthroplasty: a meta-analysis of randomised trials. Int Orthop 35:3–8PubMedCrossRef Xue D, Zheng Q, Li H, Qian S, Zhang B, Pan Z (2011) Selective COX-2 inhibitor versus nonselective COX-1 and COX-2 inhibitor in the prevention of heterotopic ossification after total hip arthroplasty: a meta-analysis of randomised trials. Int Orthop 35:3–8PubMedCrossRef
31.
go back to reference Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, for the QUOROM Group (1999) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement. Lancet 354:1896–1900PubMedCrossRef Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, for the QUOROM Group (1999) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement. Lancet 354:1896–1900PubMedCrossRef
32.
go back to reference Moher D, Liberati A, Tetzlaff J, Altman DG for the PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6: e1000097 Moher D, Liberati A, Tetzlaff J, Altman DG for the PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6: e1000097
33.
go back to reference Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6:e1000100PubMedCrossRef Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6:e1000100PubMedCrossRef
34.
35.
go back to reference Liu M, Yang Z, Pei F, Huang F, Chen S, Xiang Z (2010) A meta-analysis of the Gamma nail and dynamic hip screw in treating peritrochanteric fractures. Int Orthop 34:323–328PubMedCrossRef Liu M, Yang Z, Pei F, Huang F, Chen S, Xiang Z (2010) A meta-analysis of the Gamma nail and dynamic hip screw in treating peritrochanteric fractures. Int Orthop 34:323–328PubMedCrossRef
36.
go back to reference Furlan A, Clarke J, Esmail R, Sinclair S, Irvin E, Bombardier C (2001) A critical review of reviews on the treatment of chronic low back pain. Spine 26:E155–E162PubMedCrossRef Furlan A, Clarke J, Esmail R, Sinclair S, Irvin E, Bombardier C (2001) A critical review of reviews on the treatment of chronic low back pain. Spine 26:E155–E162PubMedCrossRef
37.
go back to reference Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG (2007) Epidemiology and reporting characteristics of systematic reviews. PLoS Med 4:e78PubMedCrossRef Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG (2007) Epidemiology and reporting characteristics of systematic reviews. PLoS Med 4:e78PubMedCrossRef
38.
go back to reference Shea BJ, Dubé C, Moher D (2001) Assessing the quality of reports of systematic reviews: the QUOROM statement compared to other tools. In: Egger M, Smith GD, Altman DG (eds) Systematic reviews in health care: meta-analysis in context. BMJ Books, London, pp 122–139CrossRef Shea BJ, Dubé C, Moher D (2001) Assessing the quality of reports of systematic reviews: the QUOROM statement compared to other tools. In: Egger M, Smith GD, Altman DG (eds) Systematic reviews in health care: meta-analysis in context. BMJ Books, London, pp 122–139CrossRef
39.
go back to reference Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C et al (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10PubMedCrossRef Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C et al (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7:10PubMedCrossRef
40.
go back to reference Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J et al (2009) AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 62:1013–1020PubMedCrossRef Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J et al (2009) AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 62:1013–1020PubMedCrossRef
41.
go back to reference Eden J, Levit L, Berg A, Morton S (eds) (2011) Finding what works in health care: standards for systematic reviews. National Academies Press, Washington, DC Eden J, Levit L, Berg A, Morton S (eds) (2011) Finding what works in health care: standards for systematic reviews. National Academies Press, Washington, DC
42.
go back to reference Starr M, Chalmers I, Clarke M, Oxman AD (2009) The origins, evolution, and future of The Cochrane Database of Systematic Reviews. Int J Technol Assess Health Care 25(1):182–195PubMedCrossRef Starr M, Chalmers I, Clarke M, Oxman AD (2009) The origins, evolution, and future of The Cochrane Database of Systematic Reviews. Int J Technol Assess Health Care 25(1):182–195PubMedCrossRef
43.
go back to reference Higgins JPT, Green S, eds (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011 Higgins JPT, Green S, eds (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.​cochrane-handbook.​org. Accessed 14 Nov 2011
44.
go back to reference Lang TA (2004) The value of systematic reviews as research activities in medical education. Acad Med 79:1067–1072PubMedCrossRef Lang TA (2004) The value of systematic reviews as research activities in medical education. Acad Med 79:1067–1072PubMedCrossRef
Metadata
Title
Steps in the undertaking of a systematic review in orthopaedic surgery
Authors
Dario Sambunjak
Miljenko Franić
Publication date
01-03-2012
Publisher
Springer-Verlag
Published in
International Orthopaedics / Issue 3/2012
Print ISSN: 0341-2695
Electronic ISSN: 1432-5195
DOI
https://doi.org/10.1007/s00264-011-1460-y

Other articles of this Issue 3/2012

International Orthopaedics 3/2012 Go to the issue