Skip to main content
Top
Published in: Health Research Policy and Systems 1/2004

Open Access 01-12-2004 | Research

Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's 'Arthritis Research Campaign'

Authors: Stephen R Hanney, Jonathan Grant, Steven Wooding, Martin J Buxton

Published in: Health Research Policy and Systems | Issue 1/2004

Login to get access

Abstract

Background

External and internal factors are increasingly encouraging research funding bodies to demonstrate the outcomes of their research. Traditional methods of assessing research are still important, but can be merged into broader multi-dimensional categorisations of research benefits. The onus has hitherto been on public sector funding bodies, but in the UK the role of medical charities in funding research is particularly important and the Arthritis Research Campaign, the leading medical charity in its field in the UK, commissioned a study to identify the outcomes from research that it funds. This article describes the methods to be used.

Methods

A case study approach will enable narratives to be told, illuminating how research funded in the early 1990s was (or was not) translated into practice. Each study will be organised using a common structure, which, with careful selection of cases, should enable cross-case analysis to illustrate the strengths of different modes and categories of research. Three main interdependent methods will be used: documentary and literature review; semi-structured interviews; and bibliometric analysis. The evaluative framework for organising the studies was previously used for assessing the benefits from health services research. Here, it has been specifically amended for a medical charity that funds a wide range of research and is concerned to develop the careers of researchers. It was further refined in three pilot studies. The framework has two main elements. First, a multi-dimensional categorisation of benefits going from the knowledge produced in peer reviewed journal articles through to the health and potential economic gain. The second element is a logic model, which, with various stages, should provide a way of organising the studies. The stock of knowledge is important: much research, especially basic, will feed into it and influence further research rather than directly lead to health gains. The cross-case analysis will look for factors associated with outcomes.

Conclusions

The pilots confirmed the applicability of the methods for a full study which should assist the Arthritis Research Campaign to demonstrate the outcomes from its funding, and provide it with evidence to inform its own policies.
Appendix
Available only for authorised users
Literature
1.
go back to reference Buxton M, Hanney S, Packwood T, Roberts S, Youll P: Assessing benefits from Department of Health and National Health Service Research & Development. Public Money & Management. 2000, 20: 29-34. 10.1111/1467-9302.00233.CrossRef Buxton M, Hanney S, Packwood T, Roberts S, Youll P: Assessing benefits from Department of Health and National Health Service Research & Development. Public Money & Management. 2000, 20: 29-34. 10.1111/1467-9302.00233.CrossRef
2.
go back to reference Grant J, Cottrell R, Cluzeau F, Fawcett G: Evaluating "payback" on biomedical research from papers cited in clinical guidelines: applied bibliometric study. BMJ. 2000, 320: 1107-1111. 10.1136/bmj.320.7242.1107.CrossRefPubMedPubMedCentral Grant J, Cottrell R, Cluzeau F, Fawcett G: Evaluating "payback" on biomedical research from papers cited in clinical guidelines: applied bibliometric study. BMJ. 2000, 320: 1107-1111. 10.1136/bmj.320.7242.1107.CrossRefPubMedPubMedCentral
3.
go back to reference National Institutes of Health: FY 2000 Research Program Outcomes Assessment Material. 2000, Bethesda National Institutes of Health: FY 2000 Research Program Outcomes Assessment Material. 2000, Bethesda
4.
go back to reference Croxson B, Hanney S, Buxton M: Routine monitoring of performance: what makes health research and development different?. J Health Serv Res Policy. 2001, 6: 226-232. 10.1258/1355819011927530.CrossRefPubMed Croxson B, Hanney S, Buxton M: Routine monitoring of performance: what makes health research and development different?. J Health Serv Res Policy. 2001, 6: 226-232. 10.1258/1355819011927530.CrossRefPubMed
6.
go back to reference van Weel C: Biomedical science matters for people–so its impact should be better assessed. Lancet. 2002, 360: 1034-1035. 10.1016/S0140-6736(02)11175-5.CrossRefPubMed van Weel C: Biomedical science matters for people–so its impact should be better assessed. Lancet. 2002, 360: 1034-1035. 10.1016/S0140-6736(02)11175-5.CrossRefPubMed
7.
go back to reference Buxton M, Hanney S: How can payback from health services research be assessed?. J Health Serv Res Policy. 1996, 1: 35-43.PubMed Buxton M, Hanney S: How can payback from health services research be assessed?. J Health Serv Res Policy. 1996, 1: 35-43.PubMed
8.
go back to reference Buxton M, Hanney S: Evaluating the NHS research and development programme: will the programme give value for money?. J R Soc Med. 1998, 91 (Suppl 35): 2-6.PubMedPubMedCentral Buxton M, Hanney S: Evaluating the NHS research and development programme: will the programme give value for money?. J R Soc Med. 1998, 91 (Suppl 35): 2-6.PubMedPubMedCentral
10.
go back to reference Grant J, Allen L: Promoting Innovation in Biomedical Research: An Evaluation of the Wellcome Trust's Showcase Scheme. 1998, London: Unit for Policy Research in Science and Medicine, Wellcome Trust Grant J, Allen L: Promoting Innovation in Biomedical Research: An Evaluation of the Wellcome Trust's Showcase Scheme. 1998, London: Unit for Policy Research in Science and Medicine, Wellcome Trust
11.
go back to reference Grant J, Allen L: Evaluating high risk research: an assessment of the Wellcome Trust's Sir Henry Wellcome Commemorative Awards for Innovative Research. Research Evaluation. 1999, 8: 201-204.CrossRef Grant J, Allen L: Evaluating high risk research: an assessment of the Wellcome Trust's Sir Henry Wellcome Commemorative Awards for Innovative Research. Research Evaluation. 1999, 8: 201-204.CrossRef
12.
go back to reference Hanney S, Packwood T, Buxton M: Evaluating the benefits from health research and development centres: a categorization, a model, and examples of application. Evaluation: The International Journal of Theory, Research and Practice. 2000, 6: 137-160. 10.1177/13563890022209181.CrossRef Hanney S, Packwood T, Buxton M: Evaluating the benefits from health research and development centres: a categorization, a model, and examples of application. Evaluation: The International Journal of Theory, Research and Practice. 2000, 6: 137-160. 10.1177/13563890022209181.CrossRef
14.
go back to reference Mushkin S: Biomedical Research: Costs and Benefits. 1979, Cambridge, Mass: Ballinger Publishing Company Mushkin S: Biomedical Research: Costs and Benefits. 1979, Cambridge, Mass: Ballinger Publishing Company
15.
go back to reference Weisbrod B: Economics and Medical Research. 1983, Washington DC: American Enterprise Institute for Public Policy Research Weisbrod B: Economics and Medical Research. 1983, Washington DC: American Enterprise Institute for Public Policy Research
16.
go back to reference Access Economics: Exceptional Returns: The Value of Investing in Health R&D in Australia. Canberra: The Australian Society for Medical Research. 2003 Access Economics: Exceptional Returns: The Value of Investing in Health R&D in Australia. Canberra: The Australian Society for Medical Research. 2003
17.
go back to reference Funding First: Exceptional Returns: The Economic Value of America's Investment in Medical Research. New York. 2000 Funding First: Exceptional Returns: The Economic Value of America's Investment in Medical Research. New York. 2000
18.
go back to reference Wellcome Trust: Mapping the Landscape: National Biomedical Research Outputs 1988–1995. London. 1998 Wellcome Trust: Mapping the Landscape: National Biomedical Research Outputs 1988–1995. London. 1998
19.
go back to reference Lewison G, Devey ME: Bibliometric methods for the evaluation of arthritis research. Rheumatology. 1999, 38: 13-20. 10.1093/rheumatology/38.1.13.CrossRefPubMed Lewison G, Devey ME: Bibliometric methods for the evaluation of arthritis research. Rheumatology. 1999, 38: 13-20. 10.1093/rheumatology/38.1.13.CrossRefPubMed
21.
go back to reference Cozzens SE: Assessing federally-supported academic research in the United States. Research Evaluation. 2000, 9: 5-10.CrossRef Cozzens SE: Assessing federally-supported academic research in the United States. Research Evaluation. 2000, 9: 5-10.CrossRef
22.
go back to reference Medical Research Council: MRC Annual Report 1999–2000. London. 2000 Medical Research Council: MRC Annual Report 1999–2000. London. 2000
23.
go back to reference Molas-Gallart J, Tang P, Morrow S: Assessing the non-academic impact of grant-funded socio-economic research: results from a pilot study. Research Evaluation. 2000, 9: 171-182.CrossRef Molas-Gallart J, Tang P, Morrow S: Assessing the non-academic impact of grant-funded socio-economic research: results from a pilot study. Research Evaluation. 2000, 9: 171-182.CrossRef
24.
go back to reference Yin R, Moore G: Lessons on the utilization of research from nine case experiences in the natural hazards field. Knowledge in Society. 1988, 1: 25-44. Yin R, Moore G: Lessons on the utilization of research from nine case experiences in the natural hazards field. Knowledge in Society. 1988, 1: 25-44.
25.
go back to reference Jacob R, McGregor M: Assessing the impact of health technology assessment. Int J Technol Assess Health Care. 1997, 13: 68-80.CrossRefPubMed Jacob R, McGregor M: Assessing the impact of health technology assessment. Int J Technol Assess Health Care. 1997, 13: 68-80.CrossRefPubMed
26.
go back to reference Lavis J, Ross S, McLeod C, Gildiner A: Measuring the impact of health research. J Health Serv Res Policy. 2003, 8: 165-170. 10.1258/135581903322029520.CrossRefPubMed Lavis J, Ross S, McLeod C, Gildiner A: Measuring the impact of health research. J Health Serv Res Policy. 2003, 8: 165-170. 10.1258/135581903322029520.CrossRefPubMed
28.
go back to reference Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L: Changing provider behaviour. An overview of systematic reviews of interventions. Med Care. 2001, 39 (8 Suppl 2): II2-II45.PubMed Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L: Changing provider behaviour. An overview of systematic reviews of interventions. Med Care. 2001, 39 (8 Suppl 2): II2-II45.PubMed
29.
go back to reference Bozeman B, Kingsley G: R&D value-mapping: a new approach to case study-based evaluation. Journal of Technology Transfer. 1997, 22: 33-42.CrossRef Bozeman B, Kingsley G: R&D value-mapping: a new approach to case study-based evaluation. Journal of Technology Transfer. 1997, 22: 33-42.CrossRef
30.
go back to reference Yin RK: Case Study Research: Design and Methods. 1989, Newbury Park, CA: Sage, 2 Yin RK: Case Study Research: Design and Methods. 1989, Newbury Park, CA: Sage, 2
31.
go back to reference van Leeuwen TN, van der Wurff LJ, van Raan AFJ: The use of combined bibliometric methods in research funding policy. Research Evaluation. 2001, 10: 195-201.CrossRef van Leeuwen TN, van der Wurff LJ, van Raan AFJ: The use of combined bibliometric methods in research funding policy. Research Evaluation. 2001, 10: 195-201.CrossRef
32.
go back to reference van Tulder MW, Assendelft WJ, Koes BW, Bouter LM: Method guidelines for systematic reviews in the Cochrane Collaboration Back Review Group for Spinal Disorders. Spine. 1997, 22: 2323-2330. 10.1097/00007632-199710150-00001.CrossRefPubMed van Tulder MW, Assendelft WJ, Koes BW, Bouter LM: Method guidelines for systematic reviews in the Cochrane Collaboration Back Review Group for Spinal Disorders. Spine. 1997, 22: 2323-2330. 10.1097/00007632-199710150-00001.CrossRefPubMed
33.
go back to reference Cave M, Hanney S, Henkel M, Kogan M: The Use of Performance Indicators in Higher Education. 1997, London: Jessica Kingsley, 3 Cave M, Hanney S, Henkel M, Kogan M: The Use of Performance Indicators in Higher Education. 1997, London: Jessica Kingsley, 3
35.
go back to reference Lewison G, Dawson G: The effect of funding on the outputs of biomedical research. Scientometrics. 1998, 41: 17-27.CrossRef Lewison G, Dawson G: The effect of funding on the outputs of biomedical research. Scientometrics. 1998, 41: 17-27.CrossRef
38.
go back to reference Royal Netherlands Academy of Arts and Sciences: The Societal Impact of Applied Research. Towards a Quality Assessment System. Amsterdam. 2002 Royal Netherlands Academy of Arts and Sciences: The Societal Impact of Applied Research. Towards a Quality Assessment System. Amsterdam. 2002
40.
go back to reference Jones T, Hanney S, Buxton M, Burns T: What British psychiatrists read: Questionnaire survey of journal usage among clinicians. Br J Psychiatry. Jones T, Hanney S, Buxton M, Burns T: What British psychiatrists read: Questionnaire survey of journal usage among clinicians. Br J Psychiatry.
41.
go back to reference Verhorn C, Landerfeld J, Wagner D: Measuring the contribution of biomedical research to the production of health. Research Policy. 1982, 11: 3-13. 10.1016/0048-7333(82)90003-8.CrossRef Verhorn C, Landerfeld J, Wagner D: Measuring the contribution of biomedical research to the production of health. Research Policy. 1982, 11: 3-13. 10.1016/0048-7333(82)90003-8.CrossRef
42.
go back to reference Fox-Rushby JA: Disability Adjusted Life Years (DALYs) for Decision-Making?. 2002, London: Office of Health Economics Fox-Rushby JA: Disability Adjusted Life Years (DALYs) for Decision-Making?. 2002, London: Office of Health Economics
43.
go back to reference Murphy K, Topel R, Eds: Measuring the Gains from Medical Research. An Economic Approach. 2003, Chicago: The University of Chicago Press Murphy K, Topel R, Eds: Measuring the Gains from Medical Research. An Economic Approach. 2003, Chicago: The University of Chicago Press
44.
go back to reference Hanney S, Mugford M, Grant J, Buxton M: Assessing the benefits of health research: Lessons from research into the use of corticosteroids for the prevention of neonatal respiratory distress syndrome. Soc Sci Med. Hanney S, Mugford M, Grant J, Buxton M: Assessing the benefits of health research: Lessons from research into the use of corticosteroids for the prevention of neonatal respiratory distress syndrome. Soc Sci Med.
45.
go back to reference Rosenberg LE: Exceptional economic returns on investments in medical research. Med J Aust. 2002, 177: 368-371.PubMed Rosenberg LE: Exceptional economic returns on investments in medical research. Med J Aust. 2002, 177: 368-371.PubMed
46.
go back to reference Hale D, Towse A: Value of the Pharmaceutical Industry to the UK Economy. 1995, London: Office of Health Economics Hale D, Towse A: Value of the Pharmaceutical Industry to the UK Economy. 1995, London: Office of Health Economics
47.
go back to reference Gadelha C: Vaccine research, development and production in Brazil. In: Lessons in Research to Action and Policy. COHRED Working Group on Research to Action and Policy. 2000, Geneva: COHRED, 9-18. Gadelha C: Vaccine research, development and production in Brazil. In: Lessons in Research to Action and Policy. COHRED Working Group on Research to Action and Policy. 2000, Geneva: COHRED, 9-18.
48.
go back to reference Drummond MF, Davies LM, Ferris FL: Assessing the costs and benefits of medical research: the diabetic retinopathy study. Soc Sci Med. 1992, 34: 973-981. 10.1016/0277-9536(92)90128-D.CrossRefPubMed Drummond MF, Davies LM, Ferris FL: Assessing the costs and benefits of medical research: the diabetic retinopathy study. Soc Sci Med. 1992, 34: 973-981. 10.1016/0277-9536(92)90128-D.CrossRefPubMed
49.
go back to reference Klaber Moffett J, Richardson G, Sheldon TA, Maynard A: Back Pain: Its Management and Cost to Society. Centre for Health Economics Discussion Paper 129. 1995, York: University of York Klaber Moffett J, Richardson G, Sheldon TA, Maynard A: Back Pain: Its Management and Cost to Society. Centre for Health Economics Discussion Paper 129. 1995, York: University of York
50.
go back to reference Pardes H, Manton KG, Lander ES, Tolley HD, Ullian AD, Palmer H: Effects of medical research on health care and the economy. Science. 1999, 283: 36-37. 10.1126/science.283.5398.36.CrossRefPubMed Pardes H, Manton KG, Lander ES, Tolley HD, Ullian AD, Palmer H: Effects of medical research on health care and the economy. Science. 1999, 283: 36-37. 10.1126/science.283.5398.36.CrossRefPubMed
51.
go back to reference Hanney S: The potential benefits of further investment in research in nursing, midwifery and professions allied to medicine. In: Promoting research in nursing and the allied health professions. A report to Task Group 3. Technical Annexe. Higher Education Funding Council for England, Research Report 01/64. 2001, Bristol: Higher Education Funding Council for England, 81-95. Hanney S: The potential benefits of further investment in research in nursing, midwifery and professions allied to medicine. In: Promoting research in nursing and the allied health professions. A report to Task Group 3. Technical Annexe. Higher Education Funding Council for England, Research Report 01/64. 2001, Bristol: Higher Education Funding Council for England, 81-95.
52.
go back to reference Leong AL, Euller-Ziegler L: Patient advocacy and arthritis: moving forward. Bull World Health Organ. 2004, 82: 115-120.PubMedPubMedCentral Leong AL, Euller-Ziegler L: Patient advocacy and arthritis: moving forward. Bull World Health Organ. 2004, 82: 115-120.PubMedPubMedCentral
53.
go back to reference Grant J, Hanney S, Buxton M: Academic medicine: time for reinvention. Research needs researching [Letter]. BMJ. 2004, 328: 48-10.1136/bmj.328.7430.48-a.CrossRefPubMedPubMedCentral Grant J, Hanney S, Buxton M: Academic medicine: time for reinvention. Research needs researching [Letter]. BMJ. 2004, 328: 48-10.1136/bmj.328.7430.48-a.CrossRefPubMedPubMedCentral
Metadata
Title
Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's 'Arthritis Research Campaign'
Authors
Stephen R Hanney
Jonathan Grant
Steven Wooding
Martin J Buxton
Publication date
01-12-2004
Publisher
BioMed Central
Published in
Health Research Policy and Systems / Issue 1/2004
Electronic ISSN: 1478-4505
DOI
https://doi.org/10.1186/1478-4505-2-4

Other articles of this Issue 1/2004

Health Research Policy and Systems 1/2004 Go to the issue