Skip to main content
Top
Published in: Health Research Policy and Systems 1/2017

Open Access 01-12-2017 | Research

An output evaluation of a health research foundation’s enhanced grant review process for new investigators

Authors: Gregory W. Hammond, Mê-Linh Lê, Tannis Novotny, Stephanie P. B. Caligiuri, Grant N. Pierce, John Wade

Published in: Health Research Policy and Systems | Issue 1/2017

Login to get access

Abstract

Background

We assessed the ability of the Manitoba Medical Service Foundation (MMSF, a small not-for-profit foundation affiliated with Manitoba Blue Cross) to determine the best candidates for selection to receive research funding support among new researchers applying to the Research Operating Grants Programme (ROGP).

Methods

Using bibliometric and grants funding analyses, we retrospectively compared indices of academic outputs from five cohorts of MMSF-funded and not MMSF-funded applicants to the annual MMSF ROGP over 2008 to 2012, from 1 to 5 years after having received evaluation decisions from the MMSF enhanced grant review process.

Results

Those researchers funded by the MMSF competition (MMSF-funded) had a statistically significant greater number of publications, a higher h-index and greater national Tri-Council (TC) funding, versus those not selected for funding (not MMSF-funded). MMSF-funded applicants and the Manitoba research community have created a strong and rapid (within 1 to 5 years of receiving the MMSF grant) local economic return on investment associated with the MMSF ROGP that supports new investigators, of approximately nine-fold for TC grants by the principal investigator, and of 34-fold for the principal investigator on collaborative (total) TC grants.

Conclusions

The use of small amounts of seed money for competitive research grants at early stages of an MMSF-funded applicant’s career correlates with future short-term success of that applicant. The ability to correctly select promising candidates who subsequently demonstrate greater academic performance after the MMSF funding shows the selection process and the ROGP to be of merit. Multiple components may have contributed to this outcome, including a direct presentation and interview process of the candidate with five-person selection subcommittees, plus an assessment by an external reviewer (the enhanced grant review process). The selection methods used here may add value to the research grant selection processes of new researchers.
Literature
1.
go back to reference Porter ME, Kramer MR. Philanthropy's new agenda: creating value. Harv Bus Rev. 1999;77:121–31.PubMed Porter ME, Kramer MR. Philanthropy's new agenda: creating value. Harv Bus Rev. 1999;77:121–31.PubMed
2.
go back to reference van Arensbergen P, van der Weijden I, van den Besselaar P. The selection of talent as a group process. A literature review on the social dynamics of decision making in grant panels. Res Eval. 2014;23:298–311.CrossRef van Arensbergen P, van der Weijden I, van den Besselaar P. The selection of talent as a group process. A literature review on the social dynamics of decision making in grant panels. Res Eval. 2014;23:298–311.CrossRef
3.
go back to reference Panel on Return on Investment in Health Research. Making An Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research. Ottawa, ON: Canadian Academy of Health Sciences; 2009. Panel on Return on Investment in Health Research. Making An Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research. Ottawa, ON: Canadian Academy of Health Sciences; 2009.
6.
go back to reference Bornmann L, Daniel HD. The state of h index research. Is the h index the ideal way to measure research performance? EMBO Rep. 2009;10:2–6.CrossRefPubMed Bornmann L, Daniel HD. The state of h index research. Is the h index the ideal way to measure research performance? EMBO Rep. 2009;10:2–6.CrossRefPubMed
7.
go back to reference Bartneck C, Kokkelmans S. Detecting h-index manipulation through self-citation analysis. Scientometrics. 2011;87:85–98.CrossRefPubMed Bartneck C, Kokkelmans S. Detecting h-index manipulation through self-citation analysis. Scientometrics. 2011;87:85–98.CrossRefPubMed
8.
go back to reference Waltman L, van Eck NJ. The inconsistency of the h-index. J Assoc Inf Sci Technol. 2012;63:406–15.CrossRef Waltman L, van Eck NJ. The inconsistency of the h-index. J Assoc Inf Sci Technol. 2012;63:406–15.CrossRef
14.
go back to reference Scott JE, de Vries J, Iacopino AM. 25-year analysis of a dental undergraduate Research training program (BSc Dent) at the University of Manitoba Faculty of Dentistry. J Dent Res. 2008;87:1085–8.CrossRefPubMed Scott JE, de Vries J, Iacopino AM. 25-year analysis of a dental undergraduate Research training program (BSc Dent) at the University of Manitoba Faculty of Dentistry. J Dent Res. 2008;87:1085–8.CrossRefPubMed
15.
go back to reference Gerrard JM, Fish I, Tate R, Fish DG. Evaluation of the careers of graduates of the University of Manitoba's BSc (Medicine) program. CMAJ. 1988;139:1063–8.PubMedPubMedCentral Gerrard JM, Fish I, Tate R, Fish DG. Evaluation of the careers of graduates of the University of Manitoba's BSc (Medicine) program. CMAJ. 1988;139:1063–8.PubMedPubMedCentral
20.
go back to reference Olbrecht M, Bornmann L. Panel peer review of grant applications: what do we know from research in social psychology on judgment and decision-making in groups? Res Eval. 2010;19:293–304.CrossRef Olbrecht M, Bornmann L. Panel peer review of grant applications: what do we know from research in social psychology on judgment and decision-making in groups? Res Eval. 2010;19:293–304.CrossRef
Metadata
Title
An output evaluation of a health research foundation’s enhanced grant review process for new investigators
Authors
Gregory W. Hammond
Mê-Linh Lê
Tannis Novotny
Stephanie P. B. Caligiuri
Grant N. Pierce
John Wade
Publication date
01-12-2017
Publisher
BioMed Central
Published in
Health Research Policy and Systems / Issue 1/2017
Electronic ISSN: 1478-4505
DOI
https://doi.org/10.1186/s12961-017-0220-x

Other articles of this Issue 1/2017

Health Research Policy and Systems 1/2017 Go to the issue