Skip to main content
Top
Published in: Malaria Journal 1/2016

Open Access 01-12-2016 | Research

Global priorities for research and the relative importance of different research outcomes: an international Delphi survey of malaria research experts

Authors: Jo-Ann Mulligan, Lesong Conteh

Published in: Malaria Journal | Issue 1/2016

Login to get access

Abstract

Background

As global research investment increases, attention inevitably turns to assessing and measuring the outcomes and impact from research programmes. Research can have many different outcomes such as producing advances in scientific knowledge, building research capacity and, ultimately, health and broader societal benefits. The aim of this study was to test the use of a Delphi methodology as a way of gathering views from malaria research experts on research priorities and eliciting relative valuations of the different types of health research impact.

Methods

An international Delphi survey of 60 malaria research experts was used to understand views on research outcomes and priorities within malaria and across global health more widely.

Results

The study demonstrated the application of the Delphi technique to eliciting views on malaria specific research priorities, wider global health research priorities and the values assigned to different types of research impact. In terms of the most important past research successes, the development of new anti-malarial drugs and insecticide-treated bed nets were rated as the most important. When asked about research priorities for future funding, respondents ranked tackling emerging drug and insecticide resistance the highest. With respect to research impact, the panel valued research that focuses on health and health sector benefits and informing policy and product development. Contributions to scientific knowledge, although highly valued, came lower down the ranking, suggesting that efforts to move research discoveries to health products and services are valued more highly than pure advances in scientific knowledge.

Conclusions

Although the Delphi technique has been used to elicit views on research questions in global health this was the first time it has been used to assess how a group of research experts value or rank different types of research impact. The results suggest it is feasible to inject the views of a key stakeholder group into the research prioritization process and the Delphi approach is a useful tool for eliciting views on the value or importance of research impact. Future work will explore other methods for assessing and valuing research impact and test the feasibility of developing a composite tool for measuring research outcomes weighted by the values of different stakeholders.
Appendix
Available only for authorised users
Literature
1.
go back to reference Buxton M, Hanney S, Jones T. Estimating the economic value to societies of the impact of health research: a critical review. Bull World Health Organ. 2004;82:733–9.PubMedPubMedCentral Buxton M, Hanney S, Jones T. Estimating the economic value to societies of the impact of health research: a critical review. Bull World Health Organ. 2004;82:733–9.PubMedPubMedCentral
2.
go back to reference Hanney S, Buxton M, Green C, Coulson D, Raftery J. An assessment of the impact of the NHS Health Technology Assessment Programme. Health Technol Assess. 2007;11:1–180.CrossRef Hanney S, Buxton M, Green C, Coulson D, Raftery J. An assessment of the impact of the NHS Health Technology Assessment Programme. Health Technol Assess. 2007;11:1–180.CrossRef
4.
go back to reference Panel on Return on Investment in Health Research. Making an impact: a preferred framework and indicators to measure returns on investment in health research. Ottawa, ON: Canadian Academy of Health Sciences; 2009. Panel on Return on Investment in Health Research. Making an impact: a preferred framework and indicators to measure returns on investment in health research. Ottawa, ON: Canadian Academy of Health Sciences; 2009.
5.
go back to reference Banzi R, Moja L, Pistotti V, Facchini A, Liberati A. Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Res Policy Syst. 2011;9:26.CrossRefPubMedPubMedCentral Banzi R, Moja L, Pistotti V, Facchini A, Liberati A. Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Res Policy Syst. 2011;9:26.CrossRefPubMedPubMedCentral
6.
go back to reference Guthrie S, Wamaw W, Diepeveen S, Wooding S, Grant J. Measuring research: a guide to research evaluation frameworks and tools. Cambridge: RAND Europe; 2013. Guthrie S, Wamaw W, Diepeveen S, Wooding S, Grant J. Measuring research: a guide to research evaluation frameworks and tools. Cambridge: RAND Europe; 2013.
7.
go back to reference Guthrie S, Kirtley A, Garrod B, Pollitt A, Grant J, Wooding S. A decisive approach to research funding: lessons from three Retrosight studies. Cambridge: RAND Corporation; 2016.CrossRef Guthrie S, Kirtley A, Garrod B, Pollitt A, Grant J, Wooding S. A decisive approach to research funding: lessons from three Retrosight studies. Cambridge: RAND Corporation; 2016.CrossRef
8.
go back to reference Miller FA, Mentzakis E, Axler R, Lehoux P, French M, Tarride JE, et al. Do Canadian researchers and the lay public prioritize biomedical research outcomes equally? A choice experiment. Acad Med. 2013;88:519–26.CrossRefPubMed Miller FA, Mentzakis E, Axler R, Lehoux P, French M, Tarride JE, et al. Do Canadian researchers and the lay public prioritize biomedical research outcomes equally? A choice experiment. Acad Med. 2013;88:519–26.CrossRefPubMed
9.
go back to reference Turoff M. The policy Delphi. In: Linstone H, Turoff M, editors. The Delphi method: techniques and applications. Reading: Addison-Wesley Publishing Company; 2002. p. 53. Turoff M. The policy Delphi. In: Linstone H, Turoff M, editors. The Delphi method: techniques and applications. Reading: Addison-Wesley Publishing Company; 2002. p. 53.
10.
go back to reference Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12(10):1–18. Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12(10):1–18.
11.
go back to reference Okoli C, Pawlowski S. The Delphi methods as a research tool: an example, design considerations and applications. Inf Manag. 2004;42:15–29.CrossRef Okoli C, Pawlowski S. The Delphi methods as a research tool: an example, design considerations and applications. Inf Manag. 2004;42:15–29.CrossRef
12.
go back to reference Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32:1008–15.PubMed Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32:1008–15.PubMed
13.
go back to reference Fisk NM, McKee M, Atun R. Relative and absolute addressability of global disease burden in maternal and perinatal health by investment in R&D. Trop Med Int Health. 2011;16:662–8.CrossRefPubMed Fisk NM, McKee M, Atun R. Relative and absolute addressability of global disease burden in maternal and perinatal health by investment in R&D. Trop Med Int Health. 2011;16:662–8.CrossRefPubMed
14.
15.
go back to reference Lubell Y, Staedke SG, Greenwood BM, Kamya MR, Molyneux M, Newton PN, et al. Likely health outcomes for untreated acute febrile illness in the tropics in decision and economic models; a Delphi survey. PLoS ONE. 2011;6:e17439.CrossRefPubMedPubMedCentral Lubell Y, Staedke SG, Greenwood BM, Kamya MR, Molyneux M, Newton PN, et al. Likely health outcomes for untreated acute febrile illness in the tropics in decision and economic models; a Delphi survey. PLoS ONE. 2011;6:e17439.CrossRefPubMedPubMedCentral
16.
go back to reference McConalogue D, Kinn S, McNeil M, Mulligan JA. International consultation on long term global health research priorities, research capacity and research uptake in developing countries. BMC Health Res Policy Syst. (in preparation). McConalogue D, Kinn S, McNeil M, Mulligan JA. International consultation on long term global health research priorities, research capacity and research uptake in developing countries. BMC Health Res Policy Syst. (in preparation).
17.
go back to reference Health Economics Research Group, Office of Health Economics, RAND Europe. Medical Research: What’s it worth? Estimating the economic benefits from medical research in the UK. London: UK Evaluation Forum; 2008. Health Economics Research Group, Office of Health Economics, RAND Europe. Medical Research: What’s it worth? Estimating the economic benefits from medical research in the UK. London: UK Evaluation Forum; 2008.
18.
go back to reference Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1:35–43.PubMed Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1:35–43.PubMed
19.
go back to reference Wooding S, Hanney S, Buxton M, Grant J. Payback arising from research funding: evaluation of the arthritis research campaign. Rheumatology. 2005;44:1145–56.CrossRefPubMed Wooding S, Hanney S, Buxton M, Grant J. Payback arising from research funding: evaluation of the arthritis research campaign. Rheumatology. 2005;44:1145–56.CrossRefPubMed
20.
go back to reference Hanney S, Grant J, Wooding S, Buxton M. Proposed methods for reviewing the outcomes of research: the impact of funding by the UK’s arthritis research campaign. Health Res Policy Syst. 2004;2:1–11.CrossRef Hanney S, Grant J, Wooding S, Buxton M. Proposed methods for reviewing the outcomes of research: the impact of funding by the UK’s arthritis research campaign. Health Res Policy Syst. 2004;2:1–11.CrossRef
22.
go back to reference Fowler F. Survey research methods. 4th ed. Thousand Oaks: Sage; 2008. Fowler F. Survey research methods. 4th ed. Thousand Oaks: Sage; 2008.
23.
go back to reference Bradbrun N, Sudman S, Wansink B. Asking questions: the definitive guide to questionnaire design—for market research, political polls, and social and health questionnaires. San Francisco: Jossey-Bass; 2004. Bradbrun N, Sudman S, Wansink B. Asking questions: the definitive guide to questionnaire design—for market research, political polls, and social and health questionnaires. San Francisco: Jossey-Bass; 2004.
24.
go back to reference Scheibe M, Skutsch M, Schofer J. Experiments in Delphi methodology. In: Linstone H, Turoff M, editors. The Delphi method: techniques and applications. Reading: Addison-Wesley Publishing Company; 1975. p. 257–81. Scheibe M, Skutsch M, Schofer J. Experiments in Delphi methodology. In: Linstone H, Turoff M, editors. The Delphi method: techniques and applications. Reading: Addison-Wesley Publishing Company; 1975. p. 257–81.
25.
go back to reference Altschuld JW, Thomas PM. Considerations in the application of a modified scree test for Delphi survey data. Eval Rev. 1991;15:179–88.CrossRef Altschuld JW, Thomas PM. Considerations in the application of a modified scree test for Delphi survey data. Eval Rev. 1991;15:179–88.CrossRef
26.
go back to reference Prime Minister warns of global threat to antibiotic resistance. 2014. (press release). Prime Minister warns of global threat to antibiotic resistance. 2014. (press release).
27.
go back to reference WHO. The evolving threat of antimicrobial resistance: options for action. Geneva: World Health Organization; 2012. WHO. The evolving threat of antimicrobial resistance: options for action. Geneva: World Health Organization; 2012.
28.
go back to reference Alleyne G, Binagwaho A, Haines A, Jahan S, Nugent R, Rojhani A, et al. Embedding non-communicable diseases in the post-2015 development agenda. Lancet. 2013;381:566–74.CrossRefPubMed Alleyne G, Binagwaho A, Haines A, Jahan S, Nugent R, Rojhani A, et al. Embedding non-communicable diseases in the post-2015 development agenda. Lancet. 2013;381:566–74.CrossRefPubMed
29.
go back to reference WHO. Global status report on non-communicable diseases 2014. Geneva: World Health Organization; 2014. WHO. Global status report on non-communicable diseases 2014. Geneva: World Health Organization; 2014.
31.
go back to reference Pollitt A, Potoglou D, Patil S, Burge P, Guthrie S, King S, et al. Understanding the relative valuation of research impact: a best-worst scaling experiment of the general public and biomedical and health researchers. BMJ Open. 2016;6:e010916.CrossRefPubMedPubMedCentral Pollitt A, Potoglou D, Patil S, Burge P, Guthrie S, King S, et al. Understanding the relative valuation of research impact: a best-worst scaling experiment of the general public and biomedical and health researchers. BMJ Open. 2016;6:e010916.CrossRefPubMedPubMedCentral
Metadata
Title
Global priorities for research and the relative importance of different research outcomes: an international Delphi survey of malaria research experts
Authors
Jo-Ann Mulligan
Lesong Conteh
Publication date
01-12-2016
Publisher
BioMed Central
Published in
Malaria Journal / Issue 1/2016
Electronic ISSN: 1475-2875
DOI
https://doi.org/10.1186/s12936-016-1628-4

Other articles of this Issue 1/2016

Malaria Journal 1/2016 Go to the issue