Top

Published in:

01-02-2018 | ORIGINAL ARTICLE

Optimal literature search for systematic reviews in surgery

Authors: Käthe Goossen, Solveig Tenckhoff, Pascal Probst, Kathrin Grummich, André L. Mihaljevic, Markus W. Büchler, Markus K. Diener

Published in: Langenbeck's Archives of Surgery | Issue 1/2018

Abstract

Background

The aim of the present study was to determine empirically which electronic databases contribute best to a literature search in surgical systematic reviews.

Methods

For ten published systematic reviews, the systematic literature searches were repeated in the databases MEDLINE, Web of Science, CENTRAL, and EMBASE. On the basis of these reviews, a gold standard set of eligible articles was created. Recall (%), precision (%), unique contribution (%), and numbers needed to read (NNR) were calculated for each database, as well as for searches of citing references and of the reference lists of related systematic reviews (hand search).

Results

CENTRAL yielded the highest recall (88.4%) and precision (8.3%) for randomized controlled trials (RCT), MEDLINE for non-randomized studies (NRS; recall 92.6%, precision 5.2%). The most effective combination of two databases plus hand searching for RCT was MEDLINE/CENTRAL (98.6% recall, NNR 97). Adding EMBASE marginally increased the recall to 99.3%, but with an NNR of 152. For NRS, the most effective combination was MEDLINE/Web of Science (99.5% recall, NNR 60).

Conclusions

For surgical systematic reviews, the optimal literature search for RCT employs MEDLINE and CENTRAL. For surgical systematic reviews of NRS, Web of Science instead of CENTRAL should be searched. EMBASE does not contribute substantially to reviews with a surgical intervention.

Available only for authorised users

OCEBM Levels of Evidence Working Group*: “The Oxford 2011 levels of evidence”. Oxford Centre for Evidence-Based Medicine (*OCEBM Levels of Evidence Working Group*=Jeremy Howick ICJLL, Paul Glasziou, Trish Greenhalgh, Carl Heneghan, Alessandro Liberati, Ivan Moschetti, Bob Phillips HT, Olive Goddard and Mary Hodgkinson). http://www.cebm.net/index.aspx?o=5653. Accessed 5 Jul 2016

McKenzie S, Mailey B, Artinyan A, Kim J, Ellenhorn JD (2010) The incidence and outcomes of pancreatectomy in patients with metastatic pancreatic adenocarcinoma. JOP: Journal of the Pancreas [Electric Resource] 11(4):341–347

Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097 CrossRefPubMedPubMedCentral

Irwig L, Tosteson AN, Gatsonis C, Lau J, Colditz G, Chalmers TC, Mosteller F (1994) Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med 120(8):667–676. https://doi.org/10.7326/0003-4819-120-8-199404150-00008 CrossRefPubMed

Preston L, Carroll C, Gardois P, Paisley S, Kaltenthaler E (2015) Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking. Syst Rev 4(1):82. https://doi.org/10.1186/s13643-015-0074-7 CrossRefPubMedPubMedCentral

Betran AP, Say L, Gulmezoglu AM, Allen T, Hampson L (2005) Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. BMC Med Res Methodol 5(1):6. https://doi.org/10.1186/1471-2288-5-6 CrossRefPubMedPubMedCentral

Ogilvie D, Hamilton V, Egan M, Petticrew M (2005) Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health 59(9):804–808. https://doi.org/10.1136/jech.2005.034181 CrossRefPubMedPubMedCentral

Royle P, Bain L, Waugh N (2005) Systematic reviews of epidemiology in diabetes: finding the evidence. BMC Med Res Methodol 5(1):2. https://doi.org/10.1186/1471-2288-5-2 CrossRefPubMedPubMedCentral

Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, St John PD, Viola R, Raina P (2003) Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol 56(10):943–955. https://doi.org/10.1016/S0895-4356(03)00110-0 CrossRefPubMed

10.

Vassar M, Carr B, Kash-Holley M, DeWitt E, Koller C, Day J, Day K, Herrmann D, Holzmann M (2015) Database choices in endocrine systematic reviews. J Med Libr Assoc 103(4):189–192. https://doi.org/10.3163/1536-5050.103.4.005 CrossRefPubMedPubMedCentral

11.

Contin P, Goossen K, Grummich K, Jensen K, Schmitz-Winnenthal H, Buchler MW et al (2013) ENERgized vessel sealing systems versus CONventional hemostasis techniques in thyroid surgery—the ENERCON systematic review and network meta-analysis. Langenbeck’s Arch Surg 398(8):1039–1056. https://doi.org/10.1007/s00423-013-1137-7 CrossRef

12.

Harnoss JC, Ulrich AB, Harnoss JM, Diener MK, Buchler MW, Welsch T (2014) Use and results of consensus definitions in pancreatic surgery: a systematic review. Surgery 155(1):47–57. https://doi.org/10.1016/j.surg.2013.05.035 CrossRefPubMed

13.

Harnoss JM, Harnoss JC, Diener MK, Contin P, Ulrich AB, Buchler MW et al (2014) Portal annular pancreas: a systematic review of a clinical challenge. Pancreas 43(7):981–986. https://doi.org/10.1097/MPA.0000000000000186 CrossRefPubMedPubMedCentral

14.

Mehrabi A, Fischer L, Hafezi M, Dirlewanger A, Grenacher L, Diener MK, Fonouni H, Golriz M, Garoussi C, Fard N, Rahbari NN, Werner J, Büchler MW (2014) A systematic review of localization, surgical treatment options, and outcome of insulinoma. Pancreas 43(5):675–686. https://doi.org/10.1097/MPA.0000000000000110 CrossRefPubMed

15.

Huttner FJ, Koessler-Ebs J, Hackert T, Ulrich A, Buchler MW, Diener MK (2015) Meta-analysis of surgical outcome after enucleation versus standard resection for pancreatic neoplasms. Br J Surg 102(9):1026–1036. https://doi.org/10.1002/bjs.9819 CrossRefPubMed

16.

Huttner FJ, Tenckhoff S, Jensen K, Uhlmann L, Kulu Y, Buchler MW et al (2015) Meta-analysis of reconstruction techniques after low anterior resection for rectal cancer. Br J Surg 102(7):735–745. https://doi.org/10.1002/bjs.9782 CrossRefPubMed

17.

Klaiber U, Probst P, Knebel P, Contin P, Diener MK, Buchler MW et al (2015) Meta-analysis of complication rates for single-loop versus dual-loop (Roux-en-Y) with isolated pancreaticojejunostomy reconstruction after pancreaticoduodenectomy. Br J Surg 102(4):331–340. https://doi.org/10.1002/bjs.9703 CrossRefPubMed

18.

Loffler T, Rossion I, Goossen K, Saure D, Weitz J, Ulrich A et al (2015) Hand suture versus stapler for closure of loop ileostomy—a systematic review and meta-analysis of randomized controlled trials. Langenbeck's Arch Surg 400(2):193–205. https://doi.org/10.1007/s00423-014-1265-8 CrossRef

19.

Mehrabi A, Hafezi M, Arvin J, Esmaeilzadeh M, Garoussi C, Emami G, Kössler-Ebs J, Müller-Stich BP, Büchler MW, Hackert T, Diener MK (2015) A systematic review and meta-analysis of laparoscopic versus open distal pancreatectomy for benign and malignant lesions of the pancreas: it's time to randomize. Surgery 157(1):45–55. https://doi.org/10.1016/j.surg.2014.06.081 CrossRefPubMed

20.

Muller-Stich BP, Kenngott HG, Gondan M, Stock C, Linke GR, Fritz F et al (2015) Use of mesh in laparoscopic paraesophageal hernia repair: a meta-analysis and risk-benefit analysis. PLoS One 10(10):e0139547. https://doi.org/10.1371/journal.pone.0139547 CrossRefPubMedPubMedCentral

21.

Muller-Stich BP, Senft JD, Warschkow R, Kenngott HG, Billeter AT, Vit G et al (2015) Surgical versus medical treatment of type 2 diabetes mellitus in nonseverely obese patients: a systematic review and meta-analysis. Ann Surg 261(3):421–429. https://doi.org/10.1097/SLA.0000000000001014 CrossRefPubMed

22.

Probst P, Huttner FJ, Klaiber U, Knebel P, Ulrich A, Buchler MW et al (2015) Stapler versus scalpel resection followed by hand-sewn closure of the pancreatic remnant for distal pancreatectomy. Cochrane Database Syst Rev 11:Cd008688

23.

Huttner FJ, Fitzmaurice C, Schwarzer G, Seiler CM, Antes G, Buchler MW et al (2016) Pylorus-preserving pancreaticoduodenectomy (pp Whipple) versus pancreaticoduodenectomy (classic Whipple) for surgical treatment of periampullary and pancreatic carcinoma, Cd006053. Cochrane Database Syst Rev 2

24.

Huttner FJ, Mihaljevic AL, Hackert T, Ulrich A, Buchler MW, Diener MK (2016) Effectiveness of Tachosil((R)) in the prevention of postoperative pancreatic fistula after distal pancreatectomy: a systematic review and meta-analysis. Langenbeck’s Arch Surg 401(2):151–159. https://doi.org/10.1007/s00423-016-1382-7 CrossRef

25.

Kossler-Ebs JB, Grummich K, Jensen K, Huttner FJ, Muller-Stich B, Seiler CM et al (2016) Incisional hernia rates after laparoscopic or open abdominal surgery—a systematic review and meta-analysis. World J Surg 40(10):2319–2330. https://doi.org/10.1007/s00268-016-3520-3 CrossRefPubMed

26.

Probst P, Grummich K, Harnoss JC, Huttner FJ, Jensen K, Braun S et al (2016) Placebo-controlled trials in surgery: a systematic review and meta-analysis. Medicine (Baltimore) 95(17):e3516. https://doi.org/10.1097/MD.0000000000003516 CrossRef

27.

Probst P, Knebel P, Grummich K, Tenckhoff S, Ulrich A, Buchler MW et al (2016) Industry bias in randomized controlled trials in general and abdominal surgery: an empirical study. Ann Surg 264(1):87–92. https://doi.org/10.1097/SLA.0000000000001372 CrossRefPubMed

28.

Fraser C, Murray A, Burr J (2006) Identifying observational studies of surgical interventions in MEDLINE and EMBASE. BMC Med Res Methodol 6(1):41. https://doi.org/10.1186/1471-2288-6-41 CrossRefPubMedPubMedCentral

29.

Beyer FR, Wright K (2013) Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Inf Libr J 30(1):49–58. https://doi.org/10.1111/hir.12009 CrossRef

30.

Booth A (2010) How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care 26(4):431–435. https://doi.org/10.1017/S0266462310000966 CrossRefPubMed

31.

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J (2003) How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess 7(1):1–76PubMed

32.

Bachmann LM, Estermann P, Kronenberg C, ter Riet G (2003) Identifying diagnostic accuracy studies in EMBASE. J Med Libr Assoc 91(3):341–346PubMedPubMedCentral

33.

Journal Citation Reports® (Thomson Reuters, 2016)

34.

Bhangu A, Nepogodiev D, Gupta A, Torrance A, Singh P (2012) Systematic review and meta-analysis of outcomes following emergency surgery for Clostridium difficile colitis. Br J Surg 99(11):1501–1513. https://doi.org/10.1002/bjs.8868 CrossRefPubMed

35.

Constantinides VA, Christakis I, Touska P, Palazzo FF (2012) Systematic review and meta-analysis of retroperitoneoscopic versus laparoscopic adrenalectomy. Br J Surg 99(12):1639–1648. https://doi.org/10.1002/bjs.8921 CrossRefPubMed

36.

Fung AK, Aly EH (2012) Systematic review of single-incision laparoscopic colonic surgery. Br J Surg 99(10):1353–1364. https://doi.org/10.1002/bjs.8834 CrossRefPubMed

37.

Marimuthu K, Varadhan KK, Ljungqvist O, Lobo DN (2012) A meta-analysis of the effect of combinations of immune modulating nutrients on outcome in patients undergoing major open gastrointestinal surgery. Ann Surg 255(6):1060–1068. https://doi.org/10.1097/SLA.0b013e318252edf8 CrossRefPubMed

38.

O’Reilly EA, Burke JP, O'Connell PR (2012) A meta-analysis of surgical morbidity and recurrence after laparoscopic and open repair of primary unilateral inguinal hernia. Ann Surg 255(5):846–853. https://doi.org/10.1097/SLA.0b013e31824e96cf CrossRefPubMed

39.

Shabanzadeh DM, Sorensen LT (2012) Laparoscopic surgery compared with open surgery decreases surgical site infection in obese patients: a systematic review and meta-analysis. Ann Surg 256(6):934–945. https://doi.org/10.1097/SLA.0b013e318269a46b CrossRefPubMed

40.

Venkat R, Edil BH, Schulick RD, Lidor AO, Makary MA, Wolfgang CL (2012) Laparoscopic distal pancreatectomy is associated with significantly less overall morbidity compared to the open technique: a systematic review and meta-analysis. Ann Surg 255(6):1048–1059. https://doi.org/10.1097/SLA.0b013e318251ee09 CrossRefPubMed

41.

Xiong JJ, Altaf K, Mukherjee R, Huang W, WM H, Li A et al (2012) Systematic review and meta-analysis of outcomes after intraoperative pancreatic duct stent placement during pancreaticoduodenectomy. Br J Surg 99(8):1050–1061. https://doi.org/10.1002/bjs.8788 CrossRefPubMed

42.

Zeng YK, Yang ZL, Peng JS, Lin HS, Cai L (2012) Laparoscopy-assisted versus open distal gastrectomy for early gastric cancer: evidence from randomized and nonrandomized clinical trials. Ann Surg 256(1):39–52. https://doi.org/10.1097/SLA.0b013e3182583e2e CrossRefPubMed

43.

Trastulli S, Cirocchi R, Desiderio J, Guarino S, Santoro A, Parisi A, Noya G, Boselli C (2013) Systematic review and meta-analysis of randomized clinical trials comparing single-incision versus conventional laparoscopic cholecystectomy. Br J Surg 100(2):191–208. https://doi.org/10.1002/bjs.8937 CrossRefPubMed

44.

Boeker M, Vach W, Motschall E (2013) Google scholar as replacement for systematic literature searches: good relative recall and precision are not enough. BMC Med Res Methodol 13(1). https://doi.org/10.1186/1471-2288-13-131

45.

Levay P, Ainsworth N, Kettle R, Morgan A (2016) Identifying evidence for public health guidance: a comparison of citation searching with Web of Science and Google Scholar. Research Synthesis Methods 7(1):34–45. https://doi.org/10.1002/jrsm.1158 CrossRefPubMed

46.

Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM (2003) The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol 56(11):1129–1135. https://doi.org/10.1016/S0895-4356(03)00177-X CrossRefPubMed

47.

Hausner E, Waffenschmidt S, Kaiser T, Simon M (2012) Routine development of objectively derived search strategies. Syst Rev 1(1):19. https://doi.org/10.1186/2046-4053-1-19 CrossRefPubMedPubMedCentral

48.

Rathbone J, Hoffmann T, Glasziou P (2015) Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Syst Rev 4(1):80. https://doi.org/10.1186/s13643-015-0067-6 CrossRefPubMedPubMedCentral

49.

Shemilt I, Simon A, Hollands GJ, Marteau TM, Ogilvie D, O'Mara-Eves A et al: Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. Res Synth Methods 2014, 5(1):31–49, DOI: https://doi.org/10.1002/jrsm.1093

50.

Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH (2010) Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics 11(1):55. https://doi.org/10.1186/1471-2105-11-55 CrossRefPubMedPubMedCentral

51.

O'Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S (2015) Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev 4(1):5. https://doi.org/10.1186/2046-4053-4-5 CrossRefPubMedPubMedCentral

52.

Polisena J, Garritty C, Umscheid CA, Kamel C, Samra K, Smith J, Vosilla A (2015) Rapid review summit: an overview and initiation of a research agenda. Syst Rev 4:111. https://doi.org/10.1186/s13643-015-0111-6 CrossRefPubMedPubMedCentral

53.

Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, Perrier L, Hutton B, Moher D, Straus SE (2015) A scoping review of rapid review methods. BMC Med 13(1):224. https://doi.org/10.1186/s12916-015-0465-6 CrossRefPubMedPubMedCentral

54.

Lefebvre C, Eisinga A, McDonald S, Paul N (2008) Enhancing access to reports of randomized trials published world-wide—the contribution of EMBASE records to the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library. Emerg Themes Epidemiol 5(1):13. https://doi.org/10.1186/1742-7622-5-13 CrossRefPubMedPubMedCentral

55.

Hallam E, Plaice C (1999) An evaluation of EMBASE within the NHS: findings of the Database Access Project working partnership to extend the knowledge base of healthcare. Health Libr Rev 16(3):192–203. https://doi.org/10.1046/j.1365-2532.1999.0225a-1.x CrossRefPubMed

56.

Robinson KA (2005) Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol 58(3):320; author reply 1. https://doi.org/10.1016/j.jclinepi.2004.08.001 CrossRefPubMed

57.

Sampson M, Tetzlaff J, Urquhart C (2011) Precision of healthcare systematic review searches in a cross-sectional sample. Research Synthesis Methods 2(2):119–125. https://doi.org/10.1002/jrsm.42 CrossRefPubMed

58.

Sampson M, McGowan J (2011) Inquisitio validus index Medicus: a simple method of validating MEDLINE systematic review searches. Research Synthesis Methods 2(2):103–109. https://doi.org/10.1002/jrsm.40 CrossRefPubMed

59.

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7(1):10. https://doi.org/10.1186/1471-2288-7-10 CrossRefPubMedPubMedCentral

60.

Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C (2009) An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol 62(9):944–952. https://doi.org/10.1016/j.jclinepi.2008.10.012 CrossRefPubMed

61.

Wright K, Golder S, Rodriguez-Lopez R (2014) Citation searching: a systematic review case study of multiple risk behaviour interventions. BMC Med Res Methodol 14(1):73. https://doi.org/10.1186/1471-2288-14-73 CrossRefPubMedPubMedCentral

62.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B (2016) The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol 16(1):127. https://doi.org/10.1186/s12874-016-0232-1 CrossRefPubMedPubMedCentral

63.

Whiting P, Westwood M, Burke M, Sterne J, Glanville J (2008) Systematic reviews of test accuracy should search a range of databases to identify primary studies. J Clin Epidemiol 61(4):357–364. https://doi.org/10.1016/j.jclinepi.2007.05.013 CrossRefPubMed

64.

Reveiz L, Ospina E, Zorrilla AF (2004) Should we consider Embase in Latin America? J Clin Epidemiol 57(8):866; author reply 7-8–866; author reply 868. https://doi.org/10.1016/j.jclinepi.2004.02.005 CrossRefPubMed

65.

Egger M, Smith GD (1998) Bias in location and selection of studies. BMJ 316(7124):61–66. https://doi.org/10.1136/bmj.316.7124.61 CrossRefPubMedPubMedCentral

66.

McKibbon KA, Haynes RB, Dilks CJ, Ramsden MF, Ryan NC, Baker L et al (1990) How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches. Comput Biomed Res 23(6):583–593. https://doi.org/10.1016/0010-4809(90)90042-B CrossRefPubMed

67.

For an up-to-date list of search filters, see: The InterTASC Information Specialists’ Sub-Group Search Filter Resource, Search Filters by Study Design. https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/filters-to-identify-randomized-controlled-trials-and. Accessed 4 Jul 2016. Scottish Intercollegiate Guidelines Network (SIGN) search filters, http://www.sign.ac.uk/methodology/filters.html. Accessed 4 Jul 2016

68.

For a non-validated Web of Science RCT filter, see: University of Alberta Libraries, Systematic Reviews-Searching the Literature, Randomized Controlled Trial Filters, Web of Science. http://guides.library.ualberta.ca/c.php?g=248586&p=1655962. Accessed 4 Jul 2016

Title: Optimal literature search for systematic reviews in surgery
Authors: Käthe Goossen
Solveig Tenckhoff
Pascal Probst
Kathrin Grummich
André L. Mihaljevic
Markus W. Büchler
Markus K. Diener
Publication date: 01-02-2018
Publisher: Springer Berlin Heidelberg
Published in: Langenbeck's Archives of Surgery / Issue 1/2018
Print ISSN: 1435-2443
Electronic ISSN: 1435-2451
DOI: https://doi.org/10.1007/s00423-017-1646-x

At a glance: The STEP trials

Springer Medicine

Optimal literature search for systematic reviews in surgery

Abstract

Background

Methods

Results

Conclusions

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2018

Liver resection for HCC outside the BCLC criteria

The combination of mitomycin-induced blood cells with a temporary treatment of ciclosporin A prolongs allograft survival in vascularized composite allotransplantation

Pure laparoscopic anatomical segment VI resection using the Glissonian approach, Rouviere’s sulcus as a landmark, and a modified liver hanging maneuver (with video)

Influence of the recipient body mass index on the outcomes after kidney transplantation

Laparoscopic approaches to complicated diverticulitis

Statistical approach to quality assessment in liver transplantation