Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2020 | Research article

Novel citation-based search method for scientific literature: a validation study

Authors: A. Cecile J. W. Janssens, Marta Gwinn, J. Elaine Brockman, Kimberley Powell, Michael Goodman

Published in: BMC Medical Research Methodology | Issue 1/2020

Abstract

Background

We recently developed CoCites, a citation-based search method that is designed to be more efficient than traditional keyword-based methods. The method begins with identification of one or more highly relevant publications (query articles) and consists of two searches: the co-citation search, which ranks publications on their co-citation frequency with the query articles, and the citation search, which ranks publications on frequency of all citations that cite or are cited by the query articles.

Methods

We aimed to reproduce the literature searches of published systematic reviews and meta-analyses and assess whether CoCites retrieves all eligible articles while screening fewer titles.

Results

A total of 250 reviews were included. CoCites retrieved a median of 75% of the articles that were included in the original reviews. The percentage of retrieved articles was higher (88%) when the query articles were cited more frequently and when they had more overlap in their citations. Applying CoCites to only the highest-cited article yielded similar results. The co-citation and citation searches combined were more efficient when the review authors had screened more than 500 titles, but not when they had screened less.

Conclusions

CoCites is an efficient and accurate method for finding relevant related articles. The method uses the expert knowledge of authors to rank related articles, does not depend on keyword selection and requires no special expertise to build search queries. The method is transparent and reproducible.

Available only for authorised users

Bullers K, Howard AM, Hanson A, Kearns WD, Orriola JJ, Polo RL, Sakmar KA. It takes longer than you think: librarian time spent on systematic review tasks. J Med Libr Assoc. 2018;106(2):198–207. https://doi.org/10.5195/jmla.2018.323.CrossRefPubMedPubMedCentral

Simon M, Hausner E, Klaus SF, Dunton NE. Identifying nurse staffing research in Medline: development and testing of empirically derived search strategies with the PubMed interface. BMC Med Res Methodol. 2010;10:76. https://doi.org/10.1186/1471-2288-10-76.

Hu X, Rousseau R, Chen J. On the definition of forward and backward citation generations. J Inf Secur. 2011;5(1):27–36. https://doi.org/10.1016/j.joi.2010.07.004.CrossRef

Horsley T, Dingwall O, Sampson M. Checking reference lists to find additional studies for systematic reviews. Cochrane Database Syst Rev. 2011;8:MR000026. https://doi.org/10.1002/14651858.MR000026.pub2.

Wright K, Golder S, Rodriguez-Lopez R. Citation searching: a systematic review case study of multiple risk behaviour interventions. BMC Med Res Methodol. 2014;14:73. https://doi.org/10.1186/1471-2288-14-73.CrossRefPubMedPubMedCentral

Robinson KA, Dunn AG, Tsafnat G, Glasziou P. Citation networks of related trials are often disconnected: implications for bidirectional citation searches. J Clin Epidemiol. 2014;67(7):793–9. https://doi.org/10.1016/j.jclinepi.2013.11.015.CrossRefPubMed

Janssens AC, Gwinn M. Novel citation-based search method for scientific literature: application to meta-analyses. BMC Med Res Methodol. 2015;15:84. https://doi.org/10.1186/s12874-015-0077-z.CrossRefPubMedPubMedCentral

Small H. Co-citation in the scientific literature: a new measure of the relationship between two documents. J Am Soc Inf Sci. 1973;24(4):265–9.CrossRef

Ortuno FM, Rojas I, Andrade-Navarro MA, Fontaine J-F. Using cited references to improve the retrieval of related biomedical documents. BMC Bioinformatics. 2013;14. https://doi.org/10.1186/1471-2105-14-113.CrossRefPubMedPubMedCentral

10.

Boyack KW, Klavans R. Co-citation analysis, bibliographic coupling, and direct citation: which citation approach represents the research front most accurately? J Assoc Inf Sci Technol. 2010;61(12):2389–404. https://doi.org/10.1002/asi.21419.CrossRef

11.

Linder SK, Kamath GR, Pratt GF, Saraykar SS, Volk RJ. Citation searches are more sensitive than keyword searches to identify studies using specific measurement instruments. J Clin Epidemiol. 2015;68(4):412–7. https://doi.org/10.1016/j.jclinepi.2014.10.008.CrossRefPubMed

12.

Barlow P, McKee M, Basu S, Stuckler D. The health impact of trade and investment agreements: a quantitative systematic review and network co-citation analysis. Glob Health. 2017;13(1):13. https://doi.org/10.1186/s12992-017-0240-x.

13.

White HD, McCain KW. Visualizing a discipline: an author co-citation analysis of information science, 1972-1995. J Am Soc Inf Sci. 1998;49(4):327–55.

14.

Jankovic MP, Kaufmann M, Kindler CH. Active research fields in anesthesia: A document co-citation analysis of the anesthetic literature. Anesth Analg. 2008;106(5):1524–33. https://doi.org/10.1213/ane.0b013e31816d18a1.CrossRefPubMed

15.

van Eck NJ, Waltman L. Appropriate similarity measures for author co-citation analysis. J Assoc Inf Sci Technol. 2008;59:1653–61.CrossRef

16.

Trujillo CM, Long TM. Document co-citation analysis to enhance transdisciplinary research. Sci Adv. 2018;4(1). https://doi.org/10.1126/sciadv.1701130.CrossRefPubMedPubMedCentral

17.

Belter CW. Citation analysis as a literature search method for systematic reviews. J Assoc Inf Sci Technol. 2016;67(11):2766–77. https://doi.org/10.1002/asi.23605.CrossRef

18.

Belter CW. A relevance ranking method for citation-based search results. Scientometrics. 2017;112(2):731–46. https://doi.org/10.1007/s11192-017-2406-y.CrossRef

19.

Gipp B, Beel J. Citation Proximity Analysis (CPA) - A New Approach for Identifying Related Work Based on Co-Citation Analysis. Proceedings of Issi 2009 - 12th International Conference of the International Society for Scientometrics and Informetrics, Vol 2 2009;2:571–75.

20.

Eto M. Evaluations of context-based co-citation searching. Scientometrics. 2013;94(2):651–73. https://doi.org/10.1007/s11192-012-0756-z.CrossRef

21.

Callahan A, Hockema S, Eysenbach G. Contextual Cocitation: augmenting Cocitation analysis and its applications. J Assoc Inf Sci Technol. 2010;61(6):1130–43.

22.

Larsen B. Exploiting citation overlaps for information retrieval: generating a boomerang effect from the network of scientific papers. Scientometrics. 2002;54(2):155–78. https://doi.org/10.1023/a:1016011326300.CrossRef

23.

Hinde S, Spackman E. Bidirectional citation searching to completion: an exploration of literature searching methods. Pharmacoeconomics. 2015;33(1):5–11. https://doi.org/10.1007/s40273-014-0205-3.CrossRefPubMed

24.

Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of observational studies in epidemiology (MOOSE) group. JAMA. 2000;283(15):2008–12.CrossRefPubMed

25.

Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006–12. https://doi.org/10.1016/j.jclinepi.2009.06.005.CrossRefPubMed

26.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. https://doi.org/10.1136/bmj.b2700.CrossRefPubMedPubMedCentral

27.

Simpson GG. Mammals and the nature of continents. Am J Sci. 1943;241:1–31.CrossRef

Title: Novel citation-based search method for scientific literature: a validation study
Authors: A. Cecile J. W. Janssens
Marta Gwinn
J. Elaine Brockman
Kimberley Powell
Michael Goodman
Publication date: 01-12-2020
Publisher: BioMed Central
Published in: BMC Medical Research Methodology / Issue 1/2020
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-020-0907-5

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Novel citation-based search method for scientific literature: a validation study

Abstract

Background

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2020

Power difference in a χ2 test vs generalized linear mixed model in the presence of missing data – a simulation study

Establishment of a pediatric COVID-19 biorepository: unique considerations and opportunities for studying the impact of the COVID-19 pandemic on children

Correction to: Estimation of treatment preference effects in clinical trials when some participants are indifferent to treatment choice

Recruiting men from across the socioeconomic spectrum via GP registers and community outreach to a weight management feasibility randomised controlled trial

When complexity matters: a step-by-step guide to incorporating a complexity perspective in guideline development for public health and health system interventions

Analytical methods used in estimating the prevalence of HIV/AIDS from demographic and cross-sectional surveys with missing data: a systematic review