Skip to main content
Top
Published in: European Radiology 4/2019

01-04-2019 | Experimental

Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often?

Authors: Robert A. Frank, Anahita Dehmoobad Sharifabadi, Jean-Paul Salameh, Trevor A. McGrath, Noémie Kraaijpoel, Wilfred Dang, Nicole Li, Isabelle D. Gauthier, Mark Z. Wu, Patrick M. Bossuyt, Deborah Levine, Matthew D. F. McInnes

Published in: European Radiology | Issue 4/2019

Login to get access

Abstract

Objectives

To assess the risk of citation bias in imaging diagnostic accuracy research by evaluating whether studies with higher accuracy estimates are cited more frequently than those with lower accuracy estimates.

Methods

We searched Medline for diagnostic accuracy meta-analyses published in imaging journals from January 2005 to April 2016. Primary studies from the meta-analyses were screened; those assessing the diagnostic accuracy of an imaging test and reporting sensitivity and specificity were eligible for inclusion. Studies not indexed in Web of Science, duplicates, and inaccessible articles were excluded. Topic (modality/subspecialty), study design, sample size, journal impact factor, publication date, times cited, sensitivity, and specificity were extracted for each study. Negative binomial regression was performed to evaluate the association of citation rate (times cited per month since publication) with Youden’s index (sensitivity + specificity -1), highest sensitivity, and highest specificity, controlling for the potential confounding effects of modality, subspecialty, impact factor, study design, sample size, and source meta-analysis.

Results

There were 1016 primary studies included. A positive association between Youden’s index and citation rate was present, with a regression coefficient of 0.33 (p = 0.016). The regression coefficient for sensitivity was 0.41 (p = 0.034), and for specificity, 0.32 (p = 0.15).

Conclusion

A positive association exists between diagnostic accuracy estimates and citation rates, indicating that there is evidence of citation bias in imaging diagnostic accuracy literature. Overestimation of imaging test accuracy may contribute to patient harm from incorrect interpretation of test results.

Key Points

• Studies with higher accuracy estimates may be cited more frequently than those with lower accuracy estimates.
This citation bias could lead clinicians, reviews, and clinical practice guidelines to overestimate the accuracy of imaging tests, contributing to patient harm from incorrect interpretation of test results.
Appendix
Available only for authorised users
Literature
1.
go back to reference Duyx B, Urlings MJE, Swaen GMH, Bouter LM, Zeegers MP (2017) Scientific citations favor positive results: a systematic review and meta-analysis. J Clin Epidemiol 88:92–101CrossRefPubMed Duyx B, Urlings MJE, Swaen GMH, Bouter LM, Zeegers MP (2017) Scientific citations favor positive results: a systematic review and meta-analysis. J Clin Epidemiol 88:92–101CrossRefPubMed
2.
go back to reference Jannot AS, Agoritsas T, Gayet-Ageron A, Perneger TV (2013) Citation bias favoring statistically significant studies was present in medical research. J Clin Epidemiol 66:296–301CrossRefPubMed Jannot AS, Agoritsas T, Gayet-Ageron A, Perneger TV (2013) Citation bias favoring statistically significant studies was present in medical research. J Clin Epidemiol 66:296–301CrossRefPubMed
4.
go back to reference Seglen PO (1997) Citations and journal impact factors: questionable indicators of research quality. Allergy 52:1050–1056CrossRefPubMed Seglen PO (1997) Citations and journal impact factors: questionable indicators of research quality. Allergy 52:1050–1056CrossRefPubMed
5.
go back to reference Steneck NH (2006) Fostering integrity in research: definitions, current knowledge, and future directions. Sci Eng Ethics 12:53–74CrossRefPubMed Steneck NH (2006) Fostering integrity in research: definitions, current knowledge, and future directions. Sci Eng Ethics 12:53–74CrossRefPubMed
6.
go back to reference Hutchison BG, Oxman AD, Lloyd S (1995) Comprehensiveness and bias in reporting clinical trials. Study of reviews of pneumococcal vaccine effectiveness. Can Fam Physician 41:1356–1360PubMedPubMedCentral Hutchison BG, Oxman AD, Lloyd S (1995) Comprehensiveness and bias in reporting clinical trials. Study of reviews of pneumococcal vaccine effectiveness. Can Fam Physician 41:1356–1360PubMedPubMedCentral
8.
go back to reference Misemer BS, Platts-Mills TF, Jones CW (2016) Citation bias favoring positive clinical trials of thrombolytics for acute ischemic stroke: a cross-sectional analysis. Trials 17:473CrossRefPubMedPubMedCentral Misemer BS, Platts-Mills TF, Jones CW (2016) Citation bias favoring positive clinical trials of thrombolytics for acute ischemic stroke: a cross-sectional analysis. Trials 17:473CrossRefPubMedPubMedCentral
9.
go back to reference Kjaergard LL, Gluud C (2002) Citation bias of hepato-biliary randomized clinical trials. J Clin Epidemiol 55:407–410CrossRefPubMed Kjaergard LL, Gluud C (2002) Citation bias of hepato-biliary randomized clinical trials. J Clin Epidemiol 55:407–410CrossRefPubMed
10.
go back to reference Korevaar DA, Cohen JF, Spijker R et al (2016) Reported estimates of diagnostic accuracy in ophthalmology conference abstracts were not associated with full-text publication. J Clin Epidemiol 79:96–103CrossRefPubMedPubMedCentral Korevaar DA, Cohen JF, Spijker R et al (2016) Reported estimates of diagnostic accuracy in ophthalmology conference abstracts were not associated with full-text publication. J Clin Epidemiol 79:96–103CrossRefPubMedPubMedCentral
11.
go back to reference Korevaar DA, van Es N, Zwinderman AH, Cohen JF, Bossuyt PM (2016) Time to publication among completed diagnostic accuracy studies: associated with reported accuracy estimates. BMC Med Res Methodol 16:68CrossRefPubMedPubMedCentral Korevaar DA, van Es N, Zwinderman AH, Cohen JF, Bossuyt PM (2016) Time to publication among completed diagnostic accuracy studies: associated with reported accuracy estimates. BMC Med Res Methodol 16:68CrossRefPubMedPubMedCentral
12.
go back to reference Sharifabadi AD, Korevaar DA, McGrath TM, et al (2018) Reporting bias in imaging: higher accuracy is linked to faster publication. Eur Radiol 28:3632−3639 Sharifabadi AD, Korevaar DA, McGrath TM, et al (2018) Reporting bias in imaging: higher accuracy is linked to faster publication. Eur Radiol 28:3632−3639
13.
go back to reference Frank RA, McInnes MDF (2018) Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often? Study protocol. Retrieved from osf.io/j964x Frank RA, McInnes MDF (2018) Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often? Study protocol. Retrieved from osf.io/j964x
14.
go back to reference Frank RA, McInnes MDF, Levine D et al (2017) Are study and journal characteristics reliable indicators of ‘truth’ in imaging research? Radiology 287:215–223CrossRefPubMed Frank RA, McInnes MDF, Levine D et al (2017) Are study and journal characteristics reliable indicators of ‘truth’ in imaging research? Radiology 287:215–223CrossRefPubMed
15.
go back to reference McGrath TA, McInnes MD, Korevaar DA, Bossuyt PM (2016) Meta-analyses of diagnostic accuracy in imaging journals: analysis of pooling techniques and their effect on summary estimates of diagnostic accuracy. Radiology 281:78–85CrossRefPubMed McGrath TA, McInnes MD, Korevaar DA, Bossuyt PM (2016) Meta-analyses of diagnostic accuracy in imaging journals: analysis of pooling techniques and their effect on summary estimates of diagnostic accuracy. Radiology 281:78–85CrossRefPubMed
16.
go back to reference van Enst WA, Ochodo E, Scholten RJ, Hooft L, Leeflang MM (2014) Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study. BMC Med Res Methodol 14:70CrossRefPubMedPubMedCentral van Enst WA, Ochodo E, Scholten RJ, Hooft L, Leeflang MM (2014) Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study. BMC Med Res Methodol 14:70CrossRefPubMedPubMedCentral
17.
go back to reference Devillé WL, Bezemer PD, Bouter LM (2000) Publications on diagnostic test evaluation in family medicine journals: an optimal search strategy. J Clin Epidemiol 53:65–69CrossRefPubMed Devillé WL, Bezemer PD, Bouter LM (2000) Publications on diagnostic test evaluation in family medicine journals: an optimal search strategy. J Clin Epidemiol 53:65–69CrossRefPubMed
19.
go back to reference R Core Team (2017) R: A language and environment forstatistical computing. R Foundation for Statistical Computing, V., Austria. Available via https://www.R-project.org/. Accessed 1 Aug 2018 R Core Team (2017) R: A language and environment forstatistical computing. R Foundation for Statistical Computing, V., Austria. Available via https://​www.​R-project.​org/. Accessed 1 Aug 2018
20.
21.
go back to reference Kulkarni AV, Busse JW, Shams I (2007) Characteristics associated with citation rate of the medical literature. PLoS One 2:403CrossRef Kulkarni AV, Busse JW, Shams I (2007) Characteristics associated with citation rate of the medical literature. PLoS One 2:403CrossRef
22.
go back to reference Sawin VI, Robinson KA (2016) Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. J Clin Epidemiol 69:174–178CrossRefPubMed Sawin VI, Robinson KA (2016) Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. J Clin Epidemiol 69:174–178CrossRefPubMed
23.
go back to reference Sheth U, Simunovic N, Tornetta P 3rd, Einhorn TA, Bhandari M (2011) Poor citation of prior evidence in hip fracture trials. J Bone Joint Surg Am 93:2079–2086 Sheth U, Simunovic N, Tornetta P 3rd, Einhorn TA, Bhandari M (2011) Poor citation of prior evidence in hip fracture trials. J Bone Joint Surg Am 93:2079–2086
24.
go back to reference Etter JF, Stapleton J (2009) Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 62:831–837CrossRefPubMed Etter JF, Stapleton J (2009) Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. J Clin Epidemiol 62:831–837CrossRefPubMed
25.
go back to reference Nieminen P, Rucker G, Miettunen J, Carpenter J, Schumacher M (2007) Statistically significant papers in psychiatry were cited more often than others. J Clin Epidemiol 60:939–946CrossRefPubMed Nieminen P, Rucker G, Miettunen J, Carpenter J, Schumacher M (2007) Statistically significant papers in psychiatry were cited more often than others. J Clin Epidemiol 60:939–946CrossRefPubMed
26.
go back to reference Saha S, Saint A, Christakis DA (2003) Impact factor: a valid measure of journal quality? J Med Libr Assoc 9:42–46 Saha S, Saint A, Christakis DA (2003) Impact factor: a valid measure of journal quality? J Med Libr Assoc 9:42–46
27.
go back to reference Dilauro M, McInnes MD, Korevaar DA et al (2016) Is there an association between STARD statement adherence and citation rate? Radiology 280:62–67 Dilauro M, McInnes MD, Korevaar DA et al (2016) Is there an association between STARD statement adherence and citation rate? Radiology 280:62–67
28.
go back to reference Choi YJ, Chung MS, Koo HJ, Park JE, Yoon HM, Park SH (2016) Does the reporting quality of diagnostic test accuracy studies, as defined by STARD 2015, affect citation? Korean J Radiol 17:706–714CrossRefPubMedPubMedCentral Choi YJ, Chung MS, Koo HJ, Park JE, Yoon HM, Park SH (2016) Does the reporting quality of diagnostic test accuracy studies, as defined by STARD 2015, affect citation? Korean J Radiol 17:706–714CrossRefPubMedPubMedCentral
29.
go back to reference McInnes MDF, Moher D, Thombs BD et al (2018) Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA 319:388–396CrossRef McInnes MDF, Moher D, Thombs BD et al (2018) Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA 319:388–396CrossRef
30.
go back to reference McGrath TA, McInnes MDF, van Es N, Leeflang MMG, Korevaar DA, Bossuyt PMM (2017) Overinterpretation of research findings: evidence of “spin” in systematic reviews of diagnostic accuracy studies. Clin Chem 63:1353–1362CrossRefPubMed McGrath TA, McInnes MDF, van Es N, Leeflang MMG, Korevaar DA, Bossuyt PMM (2017) Overinterpretation of research findings: evidence of “spin” in systematic reviews of diagnostic accuracy studies. Clin Chem 63:1353–1362CrossRefPubMed
31.
go back to reference Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM (2013) Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”. Radiology 267:581–588CrossRefPubMed Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM (2013) Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”. Radiology 267:581–588CrossRefPubMed
Metadata
Title
Citation bias in imaging research: are studies with higher diagnostic accuracy estimates cited more often?
Authors
Robert A. Frank
Anahita Dehmoobad Sharifabadi
Jean-Paul Salameh
Trevor A. McGrath
Noémie Kraaijpoel
Wilfred Dang
Nicole Li
Isabelle D. Gauthier
Mark Z. Wu
Patrick M. Bossuyt
Deborah Levine
Matthew D. F. McInnes
Publication date
01-04-2019
Publisher
Springer Berlin Heidelberg
Published in
European Radiology / Issue 4/2019
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-018-5801-8

Other articles of this Issue 4/2019

European Radiology 4/2019 Go to the issue