Arthroplasty: critical scientometric analysis of current benchmarking and evaluation procedures

Excerpt

Research regarding “arthroplasty” is of increasing interest, when considering longevity and age progression in industrialized populations. It is very likely that the number of joint replacement numbers will increase enormously. Changes in publication methods (e.g., simplified online submission and use of citation programs) resulted in an enormous increase of scientific data since 1990. In this respect, it is difficult for a scientist to obtain an overview of a topic he/she is interested in. Scientometrics is a relatively new method to establish distributional aspects of publications (e.g., the most productive author) as well as the qualitative impact of scientific publications. Data were retrieved from ISI-Web after entering “arthroplasty” in the search field. Furthermore, they were analyzed according to previously published methods [1, 2]. Using large scale data analysis, scientometric approaches and density-equalizing procedures enabled us to make statements regarding the scientific impact. Since data acquisition is not terminated so far, results from 2008 and 2009 were excluded. From 1905 (given the year of the first published items) to 2007, 21,874 items were published (Fig. 1a). In order to determine the most prolific journals/authors dealing with the topic, two aspects are of particular interest: numbers of publications in a journal/released by an author dealing with this specific topic (measures whether the topic is of minor or major interest) and a journal’s Impact-Factor/author’s H-Index (indicates the quality of the publications of a journal/an author). “Clinical Orthopaedics and Related Research” and “Journal of Arthroplasty” combine these parameters best (Fig. 1b). They can be signified as the most prolific journals. “Harris W.H.” was identified as the most prolific author combining the highest publication numbers (163 publications) with the highest H-Index (38) (Fig. 1c). Regarding the origin of all publications, the USA, the UK and Germany are the most productive countries (Fig. 2a). International cooperation analysis revealed that the cooperation between the USA–Canada (221), the USA–the UK (110) and the USA–Japan (94) were the most productive ones (Fig. 2b). The average citation rate is defined by a single item’s number of citations. It can be used as an indicator of research quality, due to the fact that items with important scientific input are being followed by more citations than less important ones. By analyzing the average citation rate in a country-specific manner, Canada, the USA, Finland and Sweden had the highest rates. If one compares the number of published items and the average citation rate country-wise, the average citation rate appears disproportionately high for some countries (e.g., Finland and Sweden). This leads to the supposition that Finland and Sweden published items of major interest. However, further analysis of these citations revealed that the numbers of self-citation in those countries are disproportionately high and thus the main reason for our results. This citation bias indicates a current problem in qualitative and quantitative benchmarking. In brief, the author-specific H-Index and the journal-specific Impact-Factor were calculated using citation and publication numbers in a given time span. The tendency of increased self-citation may also increase the H-Index of an author disproportionately compared to his scientific impact [3, 4]. In addition, a general increase in citations since 1990 has lead to a continuous increase in Impact-Factors. This is usually a sign for an increase in the scientific quality of a journal [3, 4]. In this respect the value of the Impact-Factor should only be used to compare journals dealing with the same topic. Consequently the H-Index should be replaced by an author-specific factor which ignores self-citations.

…