Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2006

Open Access 01-12-2006 | Commentary

The relationship between quality of research and citation frequency

Authors: Pentti Nieminen, James Carpenter, Gerta Rucker, Martin Schumacher

Published in: BMC Medical Research Methodology | Issue 1/2006

Login to get access

Abstract

Background

Citation counts are often regarded as a measure of the utilization and contribution of published articles. The objective of this study is to assess whether statistical reporting and statistical errors in the analysis of the primary outcome are associated with the number of citations received.

Methods

We evaluated all original research articles published in 1996 in four psychiatric journals. The statistical and reporting quality of each paper was assessed and the number of citations received up to 2005 was obtained from the Web of Science database. We then examined whether the number of citations was associated with the quality of the statistical analysis and reporting.

Results

A total of 448 research papers were included in the citation analysis. Unclear or inadequate reporting of the research question and primary outcome were not statistically significantly associated with the citation counts. After adjusting for journal, extended description of statistical procedures had a positive effect on the number of citations received. Inappropriate statistical analysis did not affect the number of citations received. Adequate reporting of the primary research question, statistical methods and primary findings were all associated with the journal visibility and prestige.

Conclusion

In this cohort of published research, measures of reporting quality and appropriate statistical analysis were not associated with the number of citations. The journal in which a study is published appears to be as important as the statistical reporting quality in ensuring dissemination of published medical science.
Appendix
Available only for authorised users
Literature
1.
go back to reference Egghe L, Rousseau R: Introduction to informetrics. Quantitative methods in library, documentation and information science. 1990, Amsterdam: Elsevier Egghe L, Rousseau R: Introduction to informetrics. Quantitative methods in library, documentation and information science. 1990, Amsterdam: Elsevier
2.
go back to reference Moed HF: Citation analysis in research evaluation. 2005, Dordrecht: Springer Moed HF: Citation analysis in research evaluation. 2005, Dordrecht: Springer
3.
go back to reference Horton NJ, Switzer SS: Statistical methods in the journal. N Engl J Med. 2005, 353: 1977-1979. 10.1056/NEJM200511033531823.CrossRefPubMed Horton NJ, Switzer SS: Statistical methods in the journal. N Engl J Med. 2005, 353: 1977-1979. 10.1056/NEJM200511033531823.CrossRefPubMed
4.
go back to reference Miettunen J, Nieminen P, Isohanni I: Statistical methodology in major general psychiatric journals. Nord J Psychiatry. 2002, 56: 223-228. 10.1080/080394802317607219.CrossRefPubMed Miettunen J, Nieminen P, Isohanni I: Statistical methodology in major general psychiatric journals. Nord J Psychiatry. 2002, 56: 223-228. 10.1080/080394802317607219.CrossRefPubMed
5.
go back to reference Lang T: Twenty statistical errors even you can find in biomedical research articles. Croat Med J. 2004, 45: 361-370.PubMed Lang T: Twenty statistical errors even you can find in biomedical research articles. Croat Med J. 2004, 45: 361-370.PubMed
6.
go back to reference Altman DG: Poor-quality medical research: what can journals do?. JAMA. 2002, 287: 2765-2767. 10.1001/jama.287.21.2765.CrossRefPubMed Altman DG: Poor-quality medical research: what can journals do?. JAMA. 2002, 287: 2765-2767. 10.1001/jama.287.21.2765.CrossRefPubMed
7.
go back to reference Jamart J: Statistical tests in medical research. Acta Oncol. 1992, 37: 723-727.CrossRef Jamart J: Statistical tests in medical research. Acta Oncol. 1992, 37: 723-727.CrossRef
8.
go back to reference Altman DG: Statistical reviewing for medical journals. Stat Med. 1998, 17: 2661-2674. 10.1002/(SICI)1097-0258(19981215)17:23<2661::AID-SIM33>3.0.CO;2-B.CrossRefPubMed Altman DG: Statistical reviewing for medical journals. Stat Med. 1998, 17: 2661-2674. 10.1002/(SICI)1097-0258(19981215)17:23<2661::AID-SIM33>3.0.CO;2-B.CrossRefPubMed
9.
go back to reference Lang T, Secic M: How to report statistics in medicine. 1997, Philadelphia: American College of Physicians Lang T, Secic M: How to report statistics in medicine. 1997, Philadelphia: American College of Physicians
10.
go back to reference Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ: Publication and related biases. Health Technol Assess. 2000, 4 (10): Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ: Publication and related biases. Health Technol Assess. 2000, 4 (10):
11.
go back to reference Glänzel W, Thjis B, Schlemmer B: A bibliometric approach to the author self-citations in scientific communication. Scientometrics. 2004, 59: 63-77. 10.1023/B:SCIE.0000018535.99885.e9.CrossRef Glänzel W, Thjis B, Schlemmer B: A bibliometric approach to the author self-citations in scientific communication. Scientometrics. 2004, 59: 63-77. 10.1023/B:SCIE.0000018535.99885.e9.CrossRef
12.
go back to reference Moher D, Schulz KF, Altman DG: The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Medical Research Methodology. 2001, 1: 2-10.1186/1471-2288-1-2.CrossRefPubMedPubMedCentral Moher D, Schulz KF, Altman DG: The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Medical Research Methodology. 2001, 1: 2-10.1186/1471-2288-1-2.CrossRefPubMedPubMedCentral
14.
go back to reference West R, Mcllwaine A: What do citation counts count for in the field of addiction? An empirical evaluation of citation counts and their link with peer ratings of quality. Addiction. 2002, 97: 501-504. 10.1046/j.1360-0443.2002.00104.x.CrossRefPubMed West R, Mcllwaine A: What do citation counts count for in the field of addiction? An empirical evaluation of citation counts and their link with peer ratings of quality. Addiction. 2002, 97: 501-504. 10.1046/j.1360-0443.2002.00104.x.CrossRefPubMed
15.
go back to reference Callaham M, Wears RL, Weber E: Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA. 2002, 287: 2847-2850. 10.1001/jama.287.21.2847.CrossRefPubMed Callaham M, Wears RL, Weber E: Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA. 2002, 287: 2847-2850. 10.1001/jama.287.21.2847.CrossRefPubMed
16.
go back to reference Johnson T: Clinical trials in psychiatry: background and statistical perspective. Stat Methods Med Res. 1998, 7: 209-234. 10.1191/096228098668896157.CrossRefPubMed Johnson T: Clinical trials in psychiatry: background and statistical perspective. Stat Methods Med Res. 1998, 7: 209-234. 10.1191/096228098668896157.CrossRefPubMed
17.
go back to reference Goodman SN, Altman DG, George SL: Statistical reviewing policies of medical journals: caveat lector?. J Gen Intern Med. 1998, 13: 753-756. 10.1046/j.1525-1497.1998.00227.x.CrossRefPubMedPubMedCentral Goodman SN, Altman DG, George SL: Statistical reviewing policies of medical journals: caveat lector?. J Gen Intern Med. 1998, 13: 753-756. 10.1046/j.1525-1497.1998.00227.x.CrossRefPubMedPubMedCentral
18.
go back to reference Lee KP, Schotland M, Bacchetti P, Bero LA: Association of journal quality indicators with methodological quality of clinical research articles. JAMA. 2002, 287: 2805-2808. 10.1001/jama.287.21.2805.CrossRefPubMed Lee KP, Schotland M, Bacchetti P, Bero LA: Association of journal quality indicators with methodological quality of clinical research articles. JAMA. 2002, 287: 2805-2808. 10.1001/jama.287.21.2805.CrossRefPubMed
19.
go back to reference McGuigan SM: The use of statistics in the British Journal of Psychiatry. Br J Psychiatry. 1995, 167: 683-688.CrossRefPubMed McGuigan SM: The use of statistics in the British Journal of Psychiatry. Br J Psychiatry. 1995, 167: 683-688.CrossRefPubMed
21.
22.
go back to reference Peritz BC: On the Heuristic Value of Scientific Publications and Their Design – A Citation Analysis of Some Clinical-Trials. Scientometrics. 1994, 30: 175-186. 10.1007/BF02017221.CrossRef Peritz BC: On the Heuristic Value of Scientific Publications and Their Design – A Citation Analysis of Some Clinical-Trials. Scientometrics. 1994, 30: 175-186. 10.1007/BF02017221.CrossRef
23.
go back to reference Armitage P, Berry G, Matthews JNS: Statistical methods in medical research. 2002, Oxford: Blackwell ScienceCrossRef Armitage P, Berry G, Matthews JNS: Statistical methods in medical research. 2002, Oxford: Blackwell ScienceCrossRef
24.
25.
go back to reference Ioannidis JP: Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005, 294: 218-228. 10.1001/jama.294.2.218.CrossRefPubMed Ioannidis JP: Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005, 294: 218-228. 10.1001/jama.294.2.218.CrossRefPubMed
26.
go back to reference Altman DG, Goodman SN, Schroter S: How statistical expertise is used in medical research. JAMA. 2002, 287: 2817-2820. 10.1001/jama.287.21.2817.CrossRefPubMed Altman DG, Goodman SN, Schroter S: How statistical expertise is used in medical research. JAMA. 2002, 287: 2817-2820. 10.1001/jama.287.21.2817.CrossRefPubMed
Metadata
Title
The relationship between quality of research and citation frequency
Authors
Pentti Nieminen
James Carpenter
Gerta Rucker
Martin Schumacher
Publication date
01-12-2006
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2006
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-6-42

Other articles of this Issue 1/2006

BMC Medical Research Methodology 1/2006 Go to the issue