Top

European Journal of Orthopaedic Surgery & Traumatology

Published in:

01-05-2020 | Original Article

Publication integrity in orthopaedic journals: the self-citation in orthopaedic research (SCOR) threshold

Authors: Kavin Sundaram, Jared Warren, Hiba K. Anis, Alison K. Klika, Nicolas S. Piuzzi

Published in: European Journal of Orthopaedic Surgery & Traumatology | Issue 4/2020

Abstract

Background

The impact factor (IF) is the most commonly used bibliometric method for rating academic journals. However, the practice of journals’ self-citation may artificially elevate the IF. Additional bibliometric methods including Eigenfactor scale, SCImago Journal Ranking (SJR), and corrected IF (cIF) have been created. Comparing general-interest and specialized orthopaedic journals, the aims of this study were to assess: (1) the effect of journal´s self-citation on IF; (2) differences in bibliometric analysis; and (3) to determine thresholds for monitoring self-citation practices by defining the self-citation in orthopaedic research (SCOR) Threshold.

Methods

The journal citation reports and SCImago Journal and Country Rank databases were queried for orthopaedic journals from 1997 to 2017. The following bibliometrics were compared between general-interest and specialized journals: IF, cIF, Eigenfactor, self-citation rates, and SJR. A novel metric, the cIF ratio, was proposed to represent the relationship between a journal’s IF and cIF. Thresholds for cIF were based on statistical outliers of cIF ratio within general-interest and specialized journals were calculated. Outliers were defined as data points that were greater than the third quartile by 1.5 times the interquartile range using the last 10 years studied (2007–2017).

Results

Specialized orthopaedic journals had a higher median self-citation rates compared to general-interest journals (11.85% vs. 6.36%, p < 0.001). Overall, cIF ratio declined over study period, and general-interest journals had a lower cIF ratio than specialized journals (8.77% vs. 19.54%, p < 0.001). Overall, general-interest journals had more favourable values for the bibliometric indices studied compared to specialized journals The SCOR threshold for cIF ratio was determined as 25.4% for general-interest journals and 53.3% for specialized journals.

Conclusion

Overall, self-citation occurs at a higher rate in specialized versus general-interest orthopaedic journals. We propose the use of a cIF ratio along with the SCOR threshold as a tool to evaluate and monitor journal self-citation practices in orthopaedic research.

Garfield E (2006) The history and meaning of the journal impact factor. JAMA 295:90–93. https://doi.org/10.1001/jama.295.1.90 CrossRefPubMed

Garfield E (1972) Citation analysis as a tool in journal evaluation. Science 178:471–479CrossRef

Garfield E (1996) How can impact factors be improved? BMJ Br Med J 313:411–413CrossRef

Butler JS, Sebastian AS, Kaye ID et al (2017) Understanding traditional research impact metrics. Clin Spine Surg 30:164–166. https://doi.org/10.1097/BSD.0000000000000530 CrossRefPubMed

Brown T, Gutman SA (2018) Impact factor, eigenfactor, article influence, scopus SNIP, and SCImage journal rank of occupational therapy journals. Scand J Occup Ther. https://doi.org/10.1080/11038128.2018.1473489 CrossRefPubMed

van der Pol CB, McInnes MDF, Petrcich W et al (2015) Is quality and completeness of reporting of systematic reviews and meta-analyses published in high impact radiology journals associated with citation rates? PLoS ONE 10:e0119892. https://doi.org/10.1371/journal.pone.0119892 CrossRefPubMedPubMedCentral

Garner RM, Hirsch JA, Albuquerque FC, Fargen KM (2018) Bibliometric indices: defining academic productivity and citation rates of researchers, departments and journals. J Neurointerv Surg 10:102–106. https://doi.org/10.1136/neurintsurg-2017-013265 CrossRefPubMed

Rizkallah J, Sin DD (2010) Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS ONE 5:e10204. https://doi.org/10.1371/journal.pone.0010204 CrossRefPubMedPubMedCentral

Kurmis AP (2003) Understanding the limitations of the journal impact factor. J Bone Joint Surg Am 85:2449–2454CrossRef

10.

Brown T (2011) Journal quality metrics: options to consider other than impact factors. Am J Occup Ther 65:346–350CrossRef

11.

Miro O, Brown AFT, Graham CA et al (2015) Relationship between category size and journals’ impact factor: implications for emergency medicine journals and researchers. Eur J Emerg Med 22:355–362. https://doi.org/10.1097/MEJ.0000000000000184 CrossRefPubMed

12.

Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314:498–502CrossRef

13.

Clarivate Analytics Title Suppressions: Editorial Expression of Concern. In: InCites J. Cit. Reports Help. http://ipscience-help.thomsonreuters.com/incitesLiveJCR/JCRGroup/titleSuppressions.html

14.

Siebelt M, Siebelt T, Pilot P et al (2010) Citation analysis of orthopaedic literature; 18 major orthopaedic journals compared for impact factor and SCImago. BMC Musculoskelet Disord. https://doi.org/10.1186/1471-2474-11-4 CrossRefPubMedPubMedCentral

15.

(1997) Science Citation Index, Journal Citation Reports. Philadelphia, Pennsylvania, USA

16.

Hoaglin DC (2003) John W. Tukey and data analysis. Stat Sci 18:311–318. https://doi.org/10.1214/ss/1076102418 CrossRef

17.

Hawkinson MP, Krueger CA, Carroll J (2018) Self-citation does not appear to artificially inflate orthopaedic journal ranking. J Surg Orthop Adv 27:131–135CrossRef

18.

Moverley R, Rankin KS, McNamara I et al (2013) Impact factors of orthopaedic journals between 2000 and 2010: trends and comparisons with other surgical specialties. Int Orthop 37:561–567. https://doi.org/10.1007/s00264-012-1769-1 CrossRefPubMedPubMedCentral

19.

Mohamed NS, Gwam CU, Etcheson JI et al (2018) Impact factors of orthopaedic journals between 2010 and 2016: trends and comparisons with other surgical specialties. Ann Transl Med 6:114. https://doi.org/10.21037/atm.2018.03.02 CrossRefPubMedPubMedCentral

20.

Fassoulaki A, Paraskeva A, Papilas K, Karabinis G (2000) Self-citations in six anaesthesia journals and their significance in determining the impact factor. Br J Anaesth 84:266–269CrossRef

21.

Miyamoto S (2018) Self-citation rate and impact factor in the field of plastic and reconstructive surgery. J Plast Surg Hand Surg 52:40–46. https://doi.org/10.1080/2000656X.2017.1319847 CrossRefPubMed

22.

Mimouni M, Segal O (2014) Self-citation rate and impact factor in ophthalmology. Ophthalmic Res 52:136–140. https://doi.org/10.1159/000366284 CrossRefPubMed

23.

Henderson AH, Upile T, Pilavakis Y, Patel NN (2016) Reporting guidelines and journal quality in otolaryngology. Clin Otolaryngol 41:461–466. https://doi.org/10.1111/coa.12546 CrossRefPubMed

24.

Yuen J (2018) Comparison of Impact Factor, Eigenfactor Metrics, and SCImago Journal Rank Indicator and h-index for Neurosurgical and Spinal Surgical Journals. World Neurosurg. https://doi.org/10.1016/j.wneu.2018.07.144 CrossRefPubMed

25.

Slobogean G, Bhandari M (2012) Introducing levels of evidence to the journal of orthopaedic trauma: implementation and future directions. J Orthop Trauma 26:127–128. https://doi.org/10.1097/BOT.0b013e318247c931 CrossRefPubMed

26.

Bergstrom CT (2007) Eigenfactor: measuring the value and prestige of scholarly journals. Coll Res Libr News 68:314–316. https://doi.org/10.5860/crln.68.5.7804 CrossRef

27.

Guerrero-Bote VP, Moya-Anegón F (2012) A further step forward in measuring journals’ scientific prestige: the SJR2 indicator. J Informetr 6:674–688. https://doi.org/10.1016/j.joi.2012.07.001 CrossRef

28.

Lancho-Barrantes BS, Guerrero-Bote VP, Moya-Anegón F (2010) What lies behind the averages and significance of citation indicators in different disciplines? J Inf Sci 36:371–382. https://doi.org/10.1177/0165551510366077 CrossRef

29.

Janssens F, Zhang L, De Moor B, Glänzel W (2009) Hybrid clustering for validation and improvement of subject-classification schemes. Inf Process Manag 45:683–702. https://doi.org/10.1016/j.ipm.2009.06.003 CrossRef

Title: Publication integrity in orthopaedic journals: the self-citation in orthopaedic research (SCOR) threshold
Authors: Kavin Sundaram
Jared Warren
Hiba K. Anis
Alison K. Klika
Nicolas S. Piuzzi
Publication date: 01-05-2020
Publisher: Springer Paris
Published in: European Journal of Orthopaedic Surgery & Traumatology / Issue 4/2020
Print ISSN: 1633-8065
Electronic ISSN: 1432-1068
DOI: https://doi.org/10.1007/s00590-019-02616-y

At a glance: The STEP trials

Springer Medicine

Publication integrity in orthopaedic journals: the self-citation in orthopaedic research (SCOR) threshold

Abstract

Background

Methods

Results

Conclusion

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 4/2020

Strategies to minimize soft tissues and septic complications in staged management of high-energy proximal tibia fractures

Lead the way or leave the way: leading a Department of Orthopedics through the COVID-19 pandemic

Full-endoscopic (bi-portal or uni-portal) versus microscopic lumbar decompression laminectomy in patients with spinal stenosis: systematic review and meta-analysis

The proton pump inhibitor, lansoprazole, prevents the development of non-traumatic osteonecrosis of the femoral head: an experimental and prospective clinical trial

The role of the anterolateral ligament in knee’s biomechanics: a case–control retrospective study

Efficacy of platelet-rich plasma in posterior lumbar interbody fusion: systematic review and meta-analysis