Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2016

Open Access 01-12-2016 | Research article

Optimizing literature search in systematic reviews – are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders?

Authors: Thomas Aagaard, Hans Lund, Carsten Juhl

Published in: BMC Medical Research Methodology | Issue 1/2016

Login to get access

Abstract

Background

When conducting systematic reviews, it is essential to perform a comprehensive literature search to identify all published studies relevant to the specific research question. The Cochrane Collaborations Methodological Expectations of Cochrane Intervention Reviews (MECIR) guidelines state that searching MEDLINE, EMBASE and CENTRAL should be considered mandatory. The aim of this study was to evaluate the MECIR recommendations to use MEDLINE, EMBASE and CENTRAL combined, and examine the yield of using these to find randomized controlled trials (RCTs) within the area of musculoskeletal disorders.

Methods

Data sources were systematic reviews published by the Cochrane Musculoskeletal Review Group, including at least five RCTs, reporting a search history, searching MEDLINE, EMBASE, CENTRAL, and adding reference- and hand-searching. Additional databases were deemed eligible if they indexed RCTs, were in English and used in more than three of the systematic reviews. Relative recall was calculated as the number of studies identified by the literature search divided by the number of eligible studies i.e. included studies in the individual systematic reviews. Finally, cumulative median recall was calculated for MEDLINE, EMBASE and CENTRAL combined followed by the databases yielding additional studies.

Results

Deemed eligible was twenty-three systematic reviews and the databases included other than MEDLINE, EMBASE and CENTRAL was AMED, CINAHL, HealthSTAR, MANTIS, OT-Seeker, PEDro, PsychINFO, SCOPUS, SportDISCUS and Web of Science. Cumulative median recall for combined searching in MEDLINE, EMBASE and CENTRAL was 88.9% and increased to 90.9% when adding 10 additional databases.

Conclusion

Searching MEDLINE, EMBASE and CENTRAL was not sufficient for identifying all effect studies on musculoskeletal disorders, but additional ten databases did only increase the median recall by 2%. It is possible that searching databases is not sufficient to identify all relevant references, and that reviewers must rely upon additional sources in their literature search. However further research is needed.
Literature
1.
Gray JAM, Booth A, Booth S, Ison E. Evidence-based healthcare. Edinburgh: Churchill Livingstone; 2001.
2.
Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, Jamtvedt G, Nortvedt MW, Christensen R, Chalmers I. Towards evidence based research. BMJ. 2016;355:i5440.CrossRefPubMed
3.
Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Chichester: John Wiley & Sons; 2008.CrossRef
4.
Chalmers I, Nylenna M. A new network to promote evidence-based research. Lancet. 2014;384(9958):1903–4.CrossRefPubMed
5.
Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, Alonso-Coello P, Djulbegovic B, Atkins D, Falck-Ytter Y, et al. GRADE guidelines: 5. Rating the quality of evidence--publication bias. J Clin Epidemiol. 2011;64(12):1277–82.CrossRefPubMed
6.
Klassen TP, Jadad AR, Moher D. Guides for reading and interpreting systematic reviews: I. Getting started. Arch Pediatr Adolesc Med. 1998;152(7):700–4.CrossRefPubMed
7.
Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D. Methodological standards for the conduct of new Cochrane Intervention Reviews. The Cochrane Unit. 2013;2:3.
8.
Bayliss SE, Davenport CF, Pennant ME. Where and how to search for information on the effectiveness of public health interventions--a case study for prevention of cardiovascular disease. Health Info Libr J. 2014;31(4):303–13.CrossRefPubMed
9.
Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: Added value or diminishing returns? Complement Ther Med. 2004;12(4):228–32.CrossRefPubMed
10.
Rollin L, Darmoni S, Caillard JF, Gehanno JF. Searching for high-quality articles about intervention studies in occupational health - What is really missed when using only the Medline database? Scand J Work Environ Health. 2010;36(6):484–7.CrossRefPubMed
11.
Halladay CW, Trikalinos TA, Schmid IT, Schmid CH, Dahabreh IJ. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol. 2015;68(9):1076–84.CrossRefPubMed
12.
Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Inj Prev. 2008;14(6):401–4.CrossRefPubMed
13.
Minozzi S, Pistotti V, Forni M. Searching for rehabilitation articles on MEDLINE and EMBASE. An example with cross-over design. Arch Phys Med Rehabil. 2000;81(6):720–2.CrossRefPubMed
14.
Murphy LS, Reinsch S, Najm WI, Dickerson VM, Seffinger MA, Adams A, Mishra SI. Spinal palpation: The challenges of information retrieval using available databases. J Manipulative Physiol Ther. 2003;26(6):374–82.CrossRefPubMed
15.
Sampson M, Cogo E, Ajiferuke I, Manheimer E, Campbell K, Daniel R, Moher D. Searching for controlled trials of complementary and alternative medicine: A comparison of 15 databases. Evid Based Complement Alternat Med. 2011;2011:858246.PubMedPubMedCentral
16.
Shariff SZ, Sontrop JM, Iansavichus AV, Haynes RB, Weir MA, Gandhi S, Cuerden MS, Garg AX. Availability of renal literature in six bibliographic databases. Clin Kidney J. 2012;5(6):610–7.CrossRefPubMedPubMedCentral
17.
Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic Databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87.CrossRefPubMed
18.
Vickers AJ. Bibliometric analysis of randomized trials in complementary medicine. Complement Ther Med. 1998;6(4):185–9.CrossRef
19.
Whiting P, Westwood M, Burke M, Sterne J, Glanville J. Systematic reviews of test accuracy should search a range of databases to identify primary studies. J Clin Epidemiol. 2008;61(4):357.CrossRefPubMed
20.
Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: Rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(4):591–603.CrossRefPubMed
21.
Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.CrossRefPubMed
22.
Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8.CrossRefPubMedPubMedCentral
23.
Slobogean GP, Verma A, Giustini D, Slobogean BL, Mulpuri K. MEDLINE, EMBASE, and Cochrane index most primary studies but not abstracts included in orthopedic meta-analyses. J Clin Epidemiol. 2009;62(12):1261–7.CrossRefPubMed
24.
25.
26.
27.
28.
29.
Harter SP. Online information retrieval : concepts, principles and techniques. Orlando: Academic; 1986.
30.
Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006;6:33.CrossRefPubMedPubMedCentral
31.
Vincent S, Greenley S, Beaven O. Clinical Evidence diagnosis: Developing a sensitive search strategy to retrieve diagnostic studies on deep vein thrombosis: a pragmatic approach. Health Info Libr J. 2003;20(3):150–9.CrossRefPubMed
32.
Michaleff ZA, Costa LOP, Moseley AM, Maher CG, Elkins MR, Herbert RD, Sherrington C. CENTRAL, PEDro, PubMed, and EMBASE are the most comprehensive databases indexing randomized controlled trials of physical therapy interventions. Phys Ther. 2011;91(2):190–7.CrossRefPubMed
33.
Watson RJD, Richardson PH. Identifying randomized controlled trials of cognitive therapy for depression: Comparing the efficiency of embase, medline and PsycINFO bibliographic databases. Psychol Psychother Theory Res Pract. 1999;72(4):535–42.
34.
Adie S, Kwan A, Naylor JM, Harris IA, Mittal R. Cryotherapy following total knee replacement. Cochrane Database Syst Rev. 2012;9, CD007911.
35.
Bartels EM, Lund H, Hagen KB, Dagfinrud H, Christensen R, Danneskiold-Samsoe B. Aquatic exercise for the treatment of knee and hip osteoarthritis. Cochrane Database Syst Rev. 2007;4, CD005523.
36.
Bellamy N, Campbell J, Robinson V, Gee T, Bourne R, Wells G. Viscosupplementation for the treatment of osteoarthritis of the knee. Cochrane Database Syst Rev. 2006;2, CD005321.
37.
Bellamy N, Campbell J, Robinson V, Gee T, Bourne R, Wells G. Intraarticular corticosteroid for treatment of osteoarthritis of the knee. Cochrane Database Syst Rev. 2006;2, CD005328.
38.
Coghlan JA, Buchbinder R, Green S, Johnston RV, Bell SN. Surgery for rotator cuff disease. Cochrane Database Syst Rev. 2008;1, CD005619.
39.
Colebatch AN, Marks JL, Edwards CJ. Safety of non-steroidal anti-inflammatory drugs, including aspirin and paracetamol (acetaminophen) in people receiving methotrexate for inflammatory arthritis (rheumatoid arthritis, ankylosing spondylitis, psoriatic arthritis, other spondyloarthritis). Cochrane Database Syst Rev. 2011;11, CD008872.
40.
Cranney A, Welch V, Adachi JD, Homik J, Shea B, Suarez-Almazor ME, Tugwell P, Wells G. Calcitonin for the treatment and prevention of corticosteroid-induced osteoporosis. Cochrane Database Syst Rev. 2000;2, CD001983.
41.
De Morton NA, Keating JL, Jeffs K. Exercise for acutely hospitalised older medical patients. Cochrane Database Syst Rev. 2007;1, CD005955.
42.
Fidelix TS, Soares BG, Trevisani VF. Diacerein for osteoarthritis. Cochrane Database Syst Rev. 2006;1, CD005117.
43.
Karjalainen K, Malmivaara A, Van Tulder M, Roine R, Jauhiainen M, Hurri H, Koes B. Multidisciplinary rehabilitation for fibromyalgia and musculoskeletal pain in working age adults. Cochrane Database Syst Rev. 2000;2, CD001984.
44.
Katchamart W, Trudeau J, Phumethum V, Bombardier C. Methotrexate monotherapy versus methotrexate combination therapy with non-biologic disease modifying anti-rheumatic drugs for rheumatoid arthritis. Cochrane Database Syst Rev. 2010;4, CD008495.
45.
Khan F, Ng L, Gonzalez S, Hale T, Turner-Stokes L. Multidisciplinary rehabilitation programmes following joint replacement at the hip and knee in chronic arthropathy. Cochrane Database Syst Rev. 2008;2, CD004957.
46.
Lethaby A, Lopez-Olivo MA, Maxwell L, Burls A, Tugwell P, Wells GA. Etanercept for the treatment of rheumatoid arthritis. Cochrane Database Syst Rev. 2013;5, CD004525.
47.
Nuesch E, Rutjes AW, Husni E, Welch V, Juni P. Oral or transdermal opioids for osteoarthritis of the knee or hip. Cochrane Database Syst Rev. 2009;4, CD003115.
48.
Osiri M, Shea B, Robinson V, Suarez-Almazor M, Strand V, Tugwell P, Wells G. Leflunomide for treating rheumatoid arthritis. Cochrane Database Syst Rev. 2003;1, CD002047.
49.
Richards BL, Whittle SL, Buchbinder R. Antidepressants for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2011;11, CD008920.
50.
Richards BL, Whittle SL, Buchbinder R. Muscle relaxants for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2012;1, CD008922.PubMed
51.
Ruiz Garcia V, Jobanputra P, Burls A, Cabello JB, Galvez Munoz JG, Saiz Cuenca ES, Fry-Smith A. Certolizumab pegol (CDP870) for rheumatoid arthritis in adults. Cochrane Database Syst Rev. 2011;2, CD007649.
52.
Rutjes AW, Nuesch E, Sterchi R, Juni P. Therapeutic ultrasound for osteoarthritis of the knee or hip. Cochrane Database Syst Rev. 2010;1, CD003132.
53.
Rutjes AW, Nuesch E, Sterchi R, Kalichman L, Hendriks E, Osiri M, Brosseau L, Reichenbach S, Juni P. Transcutaneous electrostimulation for osteoarthritis of the knee. Cochrane Database Syst Rev. 2009;4, CD002823.
54.
Wajon A, Ada L, Edmunds I. Surgery for thumb (trapeziometacarpal joint) osteoarthritis. Cochrane Database Syst Rev. 2005;4, CD004631.
55.
Whittle SL, Richards BL, Husni E, Buchbinder R. Opioid therapy for treating rheumatoid arthritis pain. Cochrane Database Syst Rev. 2011;11, CD003113.
56.
Winzenberg TM, Powell S, Shaw KA, Jones G. Vitamin D supplementation for improving bone mineral density in children. Cochrane Database Syst Rev. 2010;10, CD006944.
57.
Brosseau L, Judd MG, Marchand S, Robinson VA, Tugwell P, Wells G, Yonge K. Transcutaneous electrical nerve stimulation (TENS) for the treatment of rheumatoid arthritis in the hand. Cochrane Database Syst Rev. 2003;3, CD004377.
58.
O’Donnell S, Cranney A, Wells GA, Adachi JD, Reginster JY. Strontium ranelate for preventing and treating postmenopausal osteoporosis. Cochrane Database Syst Rev. 2006;3, CD005326.
59.
Richards BL, Whittle SL, Buchbinder R. Neuromodulators for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2012;1, CD008921.PubMed
60.
Rome K, Ashford RL, Evans A. Non-surgical interventions for paediatric pes planus. Cochrane Database Syst Rev. 2010;7, CD006311.
61.
Savoie I, Helmer D, Green CJ, Kazanjian A. Beyond Medline: reducing bias through extended systematic review search. Int J Technol Assess Health Care. 2003;19(1):168–78.CrossRefPubMed
62.
Helmer D, Savoie I, Green C, Kazanjian A. Evidence-based practice: extending the search to find material for the systematic review. Bull Med Libr Assoc. 2001;89(4):346–52.PubMedPubMedCentral
63.
Robinson KA, Dunn AG, Tsafnat G, Glasziou P. Citation networks of related trials are often disconnected: implications for bidirectional citation searches. J Clin Epidemiol. 2014;67(7):793–9.CrossRefPubMed
64.
Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, Alderson P. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7.CrossRefPubMed
65.
Kelly L, St Pierre-Hansen N. So many databases, such little clarity: Searching the literature for the topic aboriginal. Can Fam Physician. 2008;54(11):1572–1573.e1575.PubMedPubMedCentral
66.
Adams CE, Power A, Frederick K, Lefebvre C. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8.CrossRefPubMed
67.
Aker PD, McDermaid C, Opitz BG, White MW. Searching chiropractic literature: A comparison of three computerized databases. J Manip Physiol Ther. 1996;19(8):518–24.
68.
Crumley ET, Wiebe N, Cramer K, Klassen TP, Hartling L. Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review. BMC Med Res Methodol. 2005;5:24.CrossRefPubMedPubMedCentral
69.
70.
Haafkens J, Moerman C, Schuring M, Van Dijk F. Searching bibliographic databases for literature on chronic disease and work participation. Occup Med. 2006;56(1):39–45.CrossRef
71.
Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, Yao L, Yang K. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.CrossRefPubMed
72.
Lorenzetti DL, Topfer LA, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.CrossRefPubMed
73.
Lovarini M, Wallen M, Imms C. Searching for evidence in pediatric occupational therapy using free versus subscription databases: A comparison of outcomes. Phys Occup Ther Pediatr. 2006;26(3):19–38.PubMed
74.
Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, Pill R, Russell I, Stott N, Wilkinson C. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20.CrossRefPubMed
75.
Royle PL, Bain L, Waugh NR. Sources of evidence for systematic reviews of interventions in diabetes. Diabet Med. 2005;22(10):1386–93.CrossRefPubMed
76.
Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, St John PD, Viola R, Raina P. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.CrossRefPubMed
77.
Woodman J, Harden A, Thomas J, Brunton J, Kavanagh J, Stansfield C. Searching for systematic reviews of the effects of social and environmental interventions: a case study of children and obesity. J Med Libr Assoc. 2010;98(2):140–6.CrossRefPubMedPubMedCentral
78.
Wright JM, Cottrell DJ, Mir G. Searching for religion and mental health studies required health, social science, and grey literature databases. J Clin Epidemiol. 2014;67(7):800–10.CrossRefPubMed
79.
Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58.CrossRefPubMed
80.
Moseley AM, Sherrington C, Elkins MR, Herbert RD, Maher CG. Indexing of randomised controlled trials of physiotherapy interventions: a comparison of AMED, CENTRAL, CINAHL, EMBASE, Hooked on Evidence, PEDro, PsycINFO and PubMed. Physiotherapy. 2009;95(3):151–6.CrossRefPubMed
81.
Ramos-Remus C, Suarez-Almazor M, Dorgan M, Gomez-Vargas A, Russell AS. Performance of online biomedical databases in rheumatology. J Rheumatol. 1994;21(10):1912–21.PubMed
82.
Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.CrossRefPubMed
83.
Schoones JW, Vissers T. Re: “using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions”. J Clin Epidemiol. 2016.
84.
Bramer WM, Giustini D, Kramer BM. Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study. Syst Rev. 2016;5:39.CrossRefPubMedPubMedCentral
Metadata
Title
Optimizing literature search in systematic reviews – are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders?
Authors
Thomas Aagaard
Hans Lund
Carsten Juhl
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0264-6

Other articles of this Issue 1/2016

BMC Medical Research Methodology 1/2016 Go to the issue

Research Article

Performance of models for estimating absolute risk difference in multicenter trials with binary outcome

Research Article

On the use of propensity scores in case of rare exposure

Research article

Potential use of telephone surveys for non-communicable disease surveillance in developing countries: evidence from a national household survey in Lebanon

Research article

The use of rapid review methods in health technology assessments: 3 case studies

Research article

Multiple and mixed methods in formative evaluation: Is more better? Reflections from a South African study

Research article

Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology