Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2013

Open Access 01-12-2013 | Research article

Views on the peer review system of biomedical journals: an online survey of academics from high-ranking universities

Authors: Roger Chun-Man Ho, Kwok-Kei Mak, Ren Tao, Yanxia Lu, Jeffrey R Day, Fang Pan

Published in: BMC Medical Research Methodology | Issue 1/2013

Login to get access

Abstract

Background

Peer review is the major method used by biomedical journals for making the decision of publishing an article. This cross-sectional survey assesses views concerning the review system of biomedical journals among academics globally.

Methods

A total of 28,009 biomedical academics from high-ranking universities listed by the 2009 Times Higher Education Quacquarelli Symonds (THE-QS) World University Rankings were contacted by email between March 2010 and August 2010. 1,340 completed an online survey which focused on their academic background, negative experiences and views on biomedical journal peer review and the results were compared among basic scientists, clinicians and clinician scientists.

Results

Fewer than half of the respondents agreed that the peer review systems of biomedical journals were fair (48.4%), scientific (47.5%), or transparent (25.1%). Nevertheless, 58.2% of the respondents agreed that authors should remain anonymous and 64.4% agreed that reviewers should not be disclosed. Most, (67.7%) agreed to the establishment of an appeal system. The proportion of native English-speaking respondents who agreed that the “peer review system is fair” was significantly higher than for non-native respondents (p = 0.02). Similarly, the proportion of clinicians stating that the “peer review system is fair” was significantly higher than that for basic scientists and clinician-scientists (p = 0.004). For females, (β = −0.1, p = 0.03), the frequency of encountering personal attacks in reviewers’ comments (β = −0.1, p = 0.002) and the frequency of imposition of unnecessary references by reviewers (β = −0.06, p = 0.04) were independently and inversely associated with agreement that “the peer review system is fair”.

Conclusion

Academics are divided on the issue of whether the biomedical journal peer review system is fair, scientific and transparent. A majority of academics agreed with the double-blind peer review and to the establishment of an appeal system. Female academics, experience of personal attacks and imposition of unnecessary references by reviewers were related to disagreement about fairness of the peer review system of biomedical journals.
Appendix
Available only for authorised users
Literature
1.
go back to reference Bornmann L, Daniel HD: The usefulness of peer review for selecting articles for publication: a utility analysis taking as an example a high-impact journal. PLoS One. 2010, 5: e11344-10.1371/journal.pone.0011344.CrossRefPubMedPubMedCentral Bornmann L, Daniel HD: The usefulness of peer review for selecting articles for publication: a utility analysis taking as an example a high-impact journal. PLoS One. 2010, 5: e11344-10.1371/journal.pone.0011344.CrossRefPubMedPubMedCentral
2.
go back to reference Fletcher RH, Fletcher SW: The effectiveness of editorial peer review. Peer review in health sciences. Edited by: Godless FJT. 1999, London: BMJ Books, 45-55. Fletcher RH, Fletcher SW: The effectiveness of editorial peer review. Peer review in health sciences. Edited by: Godless FJT. 1999, London: BMJ Books, 45-55.
3.
go back to reference Mahoney M: Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cogn Ther Res. 1977, 1: 161-175. 10.1007/BF01173636.CrossRef Mahoney M: Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cogn Ther Res. 1977, 1: 161-175. 10.1007/BF01173636.CrossRef
4.
go back to reference Godlee FDK: Bias, subjectivity, and COI in editorial decisions. Peer review in health sciences. Edited by: Godless FJT. 1999, London: BMJ Books, 57-78. Godlee FDK: Bias, subjectivity, and COI in editorial decisions. Peer review in health sciences. Edited by: Godless FJT. 1999, London: BMJ Books, 57-78.
5.
go back to reference Wakefield AJ: Retraction–Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 2010, 375: 445-10.1016/S0140-6736(10)60175-4.CrossRef Wakefield AJ: Retraction–Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 2010, 375: 445-10.1016/S0140-6736(10)60175-4.CrossRef
6.
go back to reference Hwang WS, Roh SI, Lee BC, Kang SK, Kwon DK, Kim S, Kim SJ, Park SW, Kwon HS, Lee CK, et al: Patient-specific embryonic stem cells derived from human SCNT blastocysts. Science. 2005, 308: 1777-1783. 10.1126/science.1112286.CrossRefPubMed Hwang WS, Roh SI, Lee BC, Kang SK, Kwon DK, Kim S, Kim SJ, Park SW, Kwon HS, Lee CK, et al: Patient-specific embryonic stem cells derived from human SCNT blastocysts. Science. 2005, 308: 1777-1783. 10.1126/science.1112286.CrossRefPubMed
7.
go back to reference Khan KM, Stovitz SD, Pluim B, Cook JL, Bahr R, Arendt EA, Noakes TD: Addressing conflicts of interest and clouding of objectivity: BJSM’s “peer review: fair review” section. Br J Sports Med. 2008, 42: 79-CrossRefPubMed Khan KM, Stovitz SD, Pluim B, Cook JL, Bahr R, Arendt EA, Noakes TD: Addressing conflicts of interest and clouding of objectivity: BJSM’s “peer review: fair review” section. Br J Sports Med. 2008, 42: 79-CrossRefPubMed
8.
go back to reference Henderson M: Problems with peer review. Br Med J. 2010, 340: c1409-10.1136/bmj.c1409.CrossRef Henderson M: Problems with peer review. Br Med J. 2010, 340: c1409-10.1136/bmj.c1409.CrossRef
9.
go back to reference Wager E, Jefferson T: Shortcomings of peer review in biomedical journals. Learned Publishing. 2001, 14: 257-263. 10.1087/095315101753141356.CrossRef Wager E, Jefferson T: Shortcomings of peer review in biomedical journals. Learned Publishing. 2001, 14: 257-263. 10.1087/095315101753141356.CrossRef
10.
go back to reference Rennie D: Misconduct and journal peer review. Peer review in health sciences. Edited by: Godless FJT. 1999, London: BMJ Books, 90-99. Rennie D: Misconduct and journal peer review. Peer review in health sciences. Edited by: Godless FJT. 1999, London: BMJ Books, 90-99.
11.
go back to reference van Rooyen S, Godlee F, Evans S, Black N, Smith R: Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. Br Med J. 1999, 318: 23-27. 10.1136/bmj.318.7175.23.CrossRef van Rooyen S, Godlee F, Evans S, Black N, Smith R: Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. Br Med J. 1999, 318: 23-27. 10.1136/bmj.318.7175.23.CrossRef
12.
go back to reference Khan K: Is open peer review the fairest system? No. Br Med J. 2010, 341: c6425-10.1136/bmj.c6425.CrossRef Khan K: Is open peer review the fairest system? No. Br Med J. 2010, 341: c6425-10.1136/bmj.c6425.CrossRef
13.
go back to reference Groves T: Is open peer review the fairest system? Yes. Br Med J. 2010, 341: c6424-10.1136/bmj.c6424.CrossRef Groves T: Is open peer review the fairest system? Yes. Br Med J. 2010, 341: c6424-10.1136/bmj.c6424.CrossRef
14.
go back to reference Benos DJ, Bashari E, Chaves JM, Gaggar A, Kapoor N, LaFrance M, Mans R, Mayhew D, McGowan S, Polter A, et al: The ups and downs of peer review. Adv Physiol Educ. 2007, 31: 145-152. 10.1152/advan.00104.2006.CrossRefPubMed Benos DJ, Bashari E, Chaves JM, Gaggar A, Kapoor N, LaFrance M, Mans R, Mayhew D, McGowan S, Polter A, et al: The ups and downs of peer review. Adv Physiol Educ. 2007, 31: 145-152. 10.1152/advan.00104.2006.CrossRefPubMed
15.
go back to reference van Rooyen S, Godlee F, Evans S, Smith R, Black N: Effect of blinding and unmasking on the quality of peer review: a randomized trial. J Am Med Assoc. 1998, 280: 234-237. 10.1001/jama.280.3.234.CrossRef van Rooyen S, Godlee F, Evans S, Smith R, Black N: Effect of blinding and unmasking on the quality of peer review: a randomized trial. J Am Med Assoc. 1998, 280: 234-237. 10.1001/jama.280.3.234.CrossRef
16.
go back to reference Cho MK, Justice AC, Winker MA, Berlin JA, Waeckerle JF, Callaham ML, Rennie D: Masking author identity in peer review: what factors influence masking success? PEER Investigators. J Am Med Assoc. 1998, 280: 243-245. 10.1001/jama.280.3.243.CrossRef Cho MK, Justice AC, Winker MA, Berlin JA, Waeckerle JF, Callaham ML, Rennie D: Masking author identity in peer review: what factors influence masking success? PEER Investigators. J Am Med Assoc. 1998, 280: 243-245. 10.1001/jama.280.3.243.CrossRef
18.
go back to reference Jefferson T, Wager E, Davidoff F: Measuring the quality of editorial peer review. JAMA. 2002, 287: 2786-2790. 10.1001/jama.287.21.2786.CrossRefPubMed Jefferson T, Wager E, Davidoff F: Measuring the quality of editorial peer review. JAMA. 2002, 287: 2786-2790. 10.1001/jama.287.21.2786.CrossRefPubMed
20.
go back to reference Eysenbach G: Improving the quality of Web surveys: the checklist for reporting results of internet E-surveys (CHERRIES). J Med Internet Res. 2004, 6 (3): e34-10.2196/jmir.6.3.e34.CrossRefPubMedPubMedCentral Eysenbach G: Improving the quality of Web surveys: the checklist for reporting results of internet E-surveys (CHERRIES). J Med Internet Res. 2004, 6 (3): e34-10.2196/jmir.6.3.e34.CrossRefPubMedPubMedCentral
21.
go back to reference Scarfe WC: Translation of the scientific method-Peer review. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2010, 109: 485-487. 10.1016/j.tripleo.2010.02.001.CrossRefPubMed Scarfe WC: Translation of the scientific method-Peer review. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2010, 109: 485-487. 10.1016/j.tripleo.2010.02.001.CrossRefPubMed
23.
go back to reference Overbeke J: The state of evidence: what we know and what we don’t know about journal peer review. Peer review in health sciences. Edited by: Godless FJT. 1999, London: BMJ Books, 32-44. Overbeke J: The state of evidence: what we know and what we don’t know about journal peer review. Peer review in health sciences. Edited by: Godless FJT. 1999, London: BMJ Books, 32-44.
24.
go back to reference Pulverer B: Transparency showcases strength of peer review. Nature. 2010, 468: 29-31. 10.1038/468029a.CrossRefPubMed Pulverer B: Transparency showcases strength of peer review. Nature. 2010, 468: 29-31. 10.1038/468029a.CrossRefPubMed
26.
go back to reference Ioannidis JP, Patsopoulos NA, Kavvoura FK, Tatsioni A, Evangelou E, Kouri I, Contopoulos-Ioannidis DG, Liberopoulos G: International ranking systems for universities and institutions: a critical appraisal. BMC Med. 2007, 5: 30-10.1186/1741-7015-5-30.CrossRefPubMedPubMedCentral Ioannidis JP, Patsopoulos NA, Kavvoura FK, Tatsioni A, Evangelou E, Kouri I, Contopoulos-Ioannidis DG, Liberopoulos G: International ranking systems for universities and institutions: a critical appraisal. BMC Med. 2007, 5: 30-10.1186/1741-7015-5-30.CrossRefPubMedPubMedCentral
27.
go back to reference Zheng ML, Yang LL, Shen Y, Shu Q: Publications in ISI-indexed public health journals from mainland China, Hong Kong and Taiwan during 1999–2008. Med Sci Mon. 2011, 17: SR21-SR27. 10.12659/MSM.881832. Zheng ML, Yang LL, Shen Y, Shu Q: Publications in ISI-indexed public health journals from mainland China, Hong Kong and Taiwan during 1999–2008. Med Sci Mon. 2011, 17: SR21-SR27. 10.12659/MSM.881832.
28.
go back to reference Poschl U: Multi-stage open peer review: scientific evaluation integrating the strengths of traditional peer review with the virtues of transparency and self-regulation. Front Comput Neurosci. 2012, 6: 33-CrossRefPubMedPubMedCentral Poschl U: Multi-stage open peer review: scientific evaluation integrating the strengths of traditional peer review with the virtues of transparency and self-regulation. Front Comput Neurosci. 2012, 6: 33-CrossRefPubMedPubMedCentral
Metadata
Title
Views on the peer review system of biomedical journals: an online survey of academics from high-ranking universities
Authors
Roger Chun-Man Ho
Kwok-Kei Mak
Ren Tao
Yanxia Lu
Jeffrey R Day
Fang Pan
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2013
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-13-74

Other articles of this Issue 1/2013

BMC Medical Research Methodology 1/2013 Go to the issue