Skip to main content
Top
Published in: BMC Public Health 1/2015

Open Access 01-12-2015 | Research article

A qualitative study on the views of experts regarding the incorporation of non-health outcomes into the economic evaluations of public health interventions

Authors: Ghislaine APG van Mastrigt, Aggie TG Paulus, Marie-Jeanne Aarts, Silvia MAA Evers, Adrienne FG Alayli-Goebbels

Published in: BMC Public Health | Issue 1/2015

Login to get access

Abstract

Background

Public health interventions can impact a broad number of outcomes, including both health and non-health outcomes (NHOs). However, although it is often acknowledged that it’s important to take NHOs into account in economic evaluation studies, in practice these are often neglected. To address this issue, our study investigated expert views regarding the incorporation of NHOs into the economic evaluations of public health interventions, by means of a qualitative study.

Methods

A purposive sampling method was used to recruit the experts in the field of health economics and/or public health for this study. Twenty-two semi-structured interviews were held. After recording, the interviews were transcribed verbatim and entered in Nvivo. The data was analyzed using a thematic analysis to identify all important themes mentioned by the experts. Data collection and analysis was continued until saturation was reached. Multiple coding and validity checks were performed to further strengthen the rigour of our methodology.

Results

Based on the expert interviews, the following overarching themes were identified; Theme 1: NHOs on the individual level, direct social level and societal level. Theme 2: Pros and Cons regarding the development of a new instrument to measure NHOs. Theme 3: The most important requirements for a new questionnaire to be developed for measuring broader outcomes. Theme 4: Alternative methods which could be used for measuring and valuating NHOs in economic evaluations for public health.

Discussion

Our research findings indicate that the importance of NHOs and the need to measure them are universally accepted. Most of the experts acknowledge the importance of measuring broader outcomes and support the development of a new instrument to measure these. The experts, who do not support the development of a new instrument, question its usefulness and feasibility; i.e., they are not sure whether it is possible to valuate NHOs. Furthermore, experts have strong and sometimes conflicting views on the specific requirements of a new instrument to be developed for measuring NHOs. They did not identify a single preferred alternative method for measurement and valuation.

Conclusions

Most experts find a wide range of NHOs important and are in favor of developing a new instrument for identifying and measuring NHOs. Hence, an open discussion needs to be initiated with experts and other stakeholders about which steps need to be taken to move forward.
Appendix
Available only for authorised users
Literature
1.
go back to reference Kelly MP, Stewart E, Morgan A, Killoran A, Fischer A, Threlfall A, et al. A conceptual framework for public health: NICE’s emerging approach. Public Health. 2009;123(1):e14–20.CrossRefPubMed Kelly MP, Stewart E, Morgan A, Killoran A, Fischer A, Threlfall A, et al. A conceptual framework for public health: NICE’s emerging approach. Public Health. 2009;123(1):e14–20.CrossRefPubMed
2.
go back to reference Drummond M, Weatherly H, Claxton K, Cookson R, Ferguson B, Godfrey C, et al. Assessing the challenges of applying standard methods of economic evaluation to public health interventions. Public Health Research Consortium, York; 2007. Drummond M, Weatherly H, Claxton K, Cookson R, Ferguson B, Godfrey C, et al. Assessing the challenges of applying standard methods of economic evaluation to public health interventions. Public Health Research Consortium, York; 2007.
3.
go back to reference Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, et al. Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations. Health Policy. 2009;93(2–3):85–92.CrossRefPubMed Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, et al. Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations. Health Policy. 2009;93(2–3):85–92.CrossRefPubMed
4.
go back to reference Alayli-Goebbels AF, Evers SM, Alexeeva D, Ament AJ, de Vries NK, Tilly JC, et al. A review of economic evaluations of behavior change interventions: setting an agenda for research methods and practice. J Public Health (Oxf). 2013;36:336–44.CrossRef Alayli-Goebbels AF, Evers SM, Alexeeva D, Ament AJ, de Vries NK, Tilly JC, et al. A review of economic evaluations of behavior change interventions: setting an agenda for research methods and practice. J Public Health (Oxf). 2013;36:336–44.CrossRef
5.
go back to reference Shiell A, Hawe P. Health promotion community development and the tyranny of individualism. Health Econ. 1996;5(3):241–7.CrossRefPubMed Shiell A, Hawe P. Health promotion community development and the tyranny of individualism. Health Econ. 1996;5(3):241–7.CrossRefPubMed
6.
go back to reference Smith RD, Petticrew M. Public health evaluation in the twenty-first century: time to see the wood as well as the trees. J Public Health (Oxf). 2010;32(1):2–7.CrossRef Smith RD, Petticrew M. Public health evaluation in the twenty-first century: time to see the wood as well as the trees. J Public Health (Oxf). 2010;32(1):2–7.CrossRef
7.
go back to reference Kelly M, Morgan A, Ellis S, Younger T, Huntley J, Swann C. Evidence based public health: A review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England. Soc Sci Med. 2010;71(6):1056–62.CrossRefPubMed Kelly M, Morgan A, Ellis S, Younger T, Huntley J, Swann C. Evidence based public health: A review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England. Soc Sci Med. 2010;71(6):1056–62.CrossRefPubMed
8.
go back to reference Developing NICE guidelines: the manual: The National Institute for Health and Care Excellence Londen, UK 2014. Developing NICE guidelines: the manual: The National Institute for Health and Care Excellence Londen, UK 2014.
11.
go back to reference Caiels J, Forder J, Malley J, Netten A, Windle K. Measuring the outcomes of low-level services: Final report. In: Discussion Paper 2699. Canterbury: University of kent; 2010. Caiels J, Forder J, Malley J, Netten A, Windle K. Measuring the outcomes of low-level services: Final report. In: Discussion Paper 2699. Canterbury: University of kent; 2010.
12.
go back to reference Al-Janabi H, Peters T, Brazier J, Bryan S, Flynn T, Clemens S, et al. An investigation of the construct validity of the ICECAP-A capability measure. Qual Life Res. 2013;22(7):1831–40.CrossRefPubMed Al-Janabi H, Peters T, Brazier J, Bryan S, Flynn T, Clemens S, et al. An investigation of the construct validity of the ICECAP-A capability measure. Qual Life Res. 2013;22(7):1831–40.CrossRefPubMed
13.
go back to reference Coast J, Peters T, Natarajan L, Sproston K, Flynn T. An assessment of the construct validity of the descriptive system for the ICECAP capability measure for older people. Qual Life Res. 2008;17(7):967–76.CrossRefPubMed Coast J, Peters T, Natarajan L, Sproston K, Flynn T. An assessment of the construct validity of the descriptive system for the ICECAP capability measure for older people. Qual Life Res. 2008;17(7):967–76.CrossRefPubMed
14.
go back to reference Bailey C, Orlando R, Kinghorn P, Armour K, Perry R, Coast J. Measuring the quality of end of life using ICECAP SCM: feasibility and acceptability. BMJ Supportive & Palliative Care. 2014;4(1):112.CrossRef Bailey C, Orlando R, Kinghorn P, Armour K, Perry R, Coast J. Measuring the quality of end of life using ICECAP SCM: feasibility and acceptability. BMJ Supportive & Palliative Care. 2014;4(1):112.CrossRef
15.
go back to reference Borghi J, Jan S. Measuring the benefits of health promotion programmes: application of the contingent valuation method. Health Policy. 2008;87(2):235–48.CrossRefPubMed Borghi J, Jan S. Measuring the benefits of health promotion programmes: application of the contingent valuation method. Health Policy. 2008;87(2):235–48.CrossRefPubMed
16.
go back to reference McDaid D, Needle J. What use has been made of economic evaluation in public health?: a systematic review of the literature. In: Dawson S, Morris ZS, editors. Future public health: burdens, challenges an opportunitites. Basingstoke UK: Palgrave Macmillan; 2008. McDaid D, Needle J. What use has been made of economic evaluation in public health?: a systematic review of the literature. In: Dawson S, Morris ZS, editors. Future public health: burdens, challenges an opportunitites. Basingstoke UK: Palgrave Macmillan; 2008.
17.
go back to reference Brazier J, Ratcliffe J, Salomon JA, Tsuchiya A. Measuring and Valuing Health Benefits for Economic Evaluation Oxford. Oxford: University Press; 2007. Brazier J, Ratcliffe J, Salomon JA, Tsuchiya A. Measuring and Valuing Health Benefits for Economic Evaluation Oxford. Oxford: University Press; 2007.
18.
go back to reference Claxton K, Sculpher M, Culyer A: Mark versus Luke? Appropriate methods for the evaluation10.1186/s12889-015-2247-7 of public health interventions. CHE Research Paper 2007, 31, Centre for Health Economics, Alcuin College, University of York York, UK. Claxton K, Sculpher M, Culyer A: Mark versus Luke? Appropriate methods for the evaluation10.1186/s12889-015-2247-7 of public health interventions. CHE Research Paper 2007, 31, Centre for Health Economics, Alcuin College, University of York York, UK.
19.
go back to reference Hale J. What contribution can health economics make to health promotion? Health Promot Int. 2000;15(4):341–8.CrossRef Hale J. What contribution can health economics make to health promotion? Health Promot Int. 2000;15(4):341–8.CrossRef
20.
go back to reference Ungar WJ. Challenges in Health State Valuation in Paediatric Economic Evaluation: Are QALYs Contraindicated? Pharmacoecon. 2011;29(8):641–52.CrossRef Ungar WJ. Challenges in Health State Valuation in Paediatric Economic Evaluation: Are QALYs Contraindicated? Pharmacoecon. 2011;29(8):641–52.CrossRef
21.
go back to reference Lorgelly PK, Lawson KD, Fenwick EA, Briggs AH. Outcome measurement in economic evaluations of public health interventions: a role for the capability approach? Int J Environ Res Public Health. 2010;7(5):2274–89.CrossRefPubMedPubMedCentral Lorgelly PK, Lawson KD, Fenwick EA, Briggs AH. Outcome measurement in economic evaluations of public health interventions: a role for the capability approach? Int J Environ Res Public Health. 2010;7(5):2274–89.CrossRefPubMedPubMedCentral
22.
go back to reference Chalkidou K, Culyer A, Naidoo B, Littlejohns P. Cost-effective public health guidance: asking questions from the decision-maker's viewpoint. Health Econ. 2008;17(3):441–8.CrossRefPubMed Chalkidou K, Culyer A, Naidoo B, Littlejohns P. Cost-effective public health guidance: asking questions from the decision-maker's viewpoint. Health Econ. 2008;17(3):441–8.CrossRefPubMed
25.
go back to reference Lasch KE, Marquis P, Vigneux M, Abetz L, Arnould B, Bayliss M, et al. PRO development: rigorous qualitative research as the crucial foundation. Qual Life Res. 2010;19(8):1087–96.CrossRefPubMedPubMedCentral Lasch KE, Marquis P, Vigneux M, Abetz L, Arnould B, Bayliss M, et al. PRO development: rigorous qualitative research as the crucial foundation. Qual Life Res. 2010;19(8):1087–96.CrossRefPubMedPubMedCentral
26.
go back to reference Braun V, Clarke V. What can "thematic analysis" offer health and wellbeing researchers? Int J Qual Stud Health Well-being. 2014;9:26152.CrossRefPubMed Braun V, Clarke V. What can "thematic analysis" offer health and wellbeing researchers? Int J Qual Stud Health Well-being. 2014;9:26152.CrossRefPubMed
27.
go back to reference Armstrong D, Gosling A, Weinman J, T M. The Place of Inter-Rater Reliability in Qualitative Research: An Empirical Study. Sociology. 1997;31(3):597–606.CrossRef Armstrong D, Gosling A, Weinman J, T M. The Place of Inter-Rater Reliability in Qualitative Research: An Empirical Study. Sociology. 1997;31(3):597–606.CrossRef
29.
go back to reference Seale c, Silverman d. Ensuring rigour in qaulitative research. Eur J Pub Health. 1997;7:379–84.CrossRef Seale c, Silverman d. Ensuring rigour in qaulitative research. Eur J Pub Health. 1997;7:379–84.CrossRef
30.
go back to reference Benning TM, Alayli-Goebbels AFG, Aarts MJ, de Wit AGA, Prenger R, Braakman A, et al. Exploring Outcomes to Consider in Economic Evaluations of Health Promotion Programs: What Broader Non-Health Outcomes Matter Most? BMC Health Services Researc. 2015;15(266):1–8. Benning TM, Alayli-Goebbels AFG, Aarts MJ, de Wit AGA, Prenger R, Braakman A, et al. Exploring Outcomes to Consider in Economic Evaluations of Health Promotion Programs: What Broader Non-Health Outcomes Matter Most? BMC Health Services Researc. 2015;15(266):1–8.
31.
go back to reference Dolan P, Peasgood T, White M: Review of research on the influences on personal well-being and application to policy making. In.; 2006. Dolan P, Peasgood T, White M: Review of research on the influences on personal well-being and application to policy making. In.; 2006.
32.
go back to reference Ryan R, Deci E. On happiness and human potentials: a review of research on hedonic and eudaimonic well-being. Annu Rev Psychol. Office for National Statistics. UK. 2001;52:141–66. Ryan R, Deci E. On happiness and human potentials: a review of research on hedonic and eudaimonic well-being. Annu Rev Psychol. Office for National Statistics. UK. 2001;52:141–66.
34.
go back to reference Coast J, Smith R, Lorgelly P. Should the capability approach be applied in health economics? Health Econ. 2008;17(6):667–70.CrossRefPubMed Coast J, Smith R, Lorgelly P. Should the capability approach be applied in health economics? Health Econ. 2008;17(6):667–70.CrossRefPubMed
35.
go back to reference Coast J, Smith RD, Lorgelly P. Welfarism, extra-welfarism and capability: the spread of ideas in health economics. Soc Sci Med. 2008;67(7):1190–8.CrossRefPubMed Coast J, Smith RD, Lorgelly P. Welfarism, extra-welfarism and capability: the spread of ideas in health economics. Soc Sci Med. 2008;67(7):1190–8.CrossRefPubMed
36.
go back to reference Olsen JA, Smith RD. Theory versus practice: a review of 'willingness-to-pay' in health and health care. Health Econ. 2001;10(1):39–52.CrossRefPubMed Olsen JA, Smith RD. Theory versus practice: a review of 'willingness-to-pay' in health and health care. Health Econ. 2001;10(1):39–52.CrossRefPubMed
37.
go back to reference Ryan M, Watson V, Entwistle V. Rationalising the 'irrational': a think aloud study of discrete choice experiment responses. Health Econ. 2009;18(3):321–36.CrossRefPubMed Ryan M, Watson V, Entwistle V. Rationalising the 'irrational': a think aloud study of discrete choice experiment responses. Health Econ. 2009;18(3):321–36.CrossRefPubMed
38.
go back to reference Bayoumi AM. The measurement of contingent valuation for health economics. Pharmacoecon. 2004;22(11):691–700.CrossRef Bayoumi AM. The measurement of contingent valuation for health economics. Pharmacoecon. 2004;22(11):691–700.CrossRef
39.
40.
go back to reference Lawson KD, Kearns A, Petticrew M, Fenwick EA. Investing in health: is social housing value for money? A cost-utility analysis. J Epidemiol Community Health. 2013;67(10):829–34.CrossRefPubMed Lawson KD, Kearns A, Petticrew M, Fenwick EA. Investing in health: is social housing value for money? A cost-utility analysis. J Epidemiol Community Health. 2013;67(10):829–34.CrossRefPubMed
41.
go back to reference De Salazar I, Jackson S, Shiell A, Rice M. Guide to economic evaluation in health promotion. Washington: Pan American Health Organisation; 2007. De Salazar I, Jackson S, Shiell A, Rice M. Guide to economic evaluation in health promotion. Washington: Pan American Health Organisation; 2007.
42.
go back to reference Thokala P. Multiple criteria decision analysis for health technology assessment. In: School of Health and Related Research. UK: University of Sheffield; 2011. Thokala P. Multiple criteria decision analysis for health technology assessment. In: School of Health and Related Research. UK: University of Sheffield; 2011.
43.
go back to reference Thokala P, Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health. 2012;15(8):1172–81.CrossRefPubMed Thokala P, Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health. 2012;15(8):1172–81.CrossRefPubMed
44.
go back to reference Aarts MJ, Jeurissen MP, van Oers HA, Schuit AJ, van de Goor IA. Multi-sector policy action to create activity-friendly environments for children: a multiple-case study. Health Policy. 2011;101(1):11–9.CrossRefPubMed Aarts MJ, Jeurissen MP, van Oers HA, Schuit AJ, van de Goor IA. Multi-sector policy action to create activity-friendly environments for children: a multiple-case study. Health Policy. 2011;101(1):11–9.CrossRefPubMed
45.
go back to reference Banke-Thomas AO, Madaj B, Charles A, van den Broek N. Social Return on Investment (SROI) methodology to account for value for money of public health interventions: a systematic review. BMC Public Health. 2015;15:582.CrossRefPubMedPubMedCentral Banke-Thomas AO, Madaj B, Charles A, van den Broek N. Social Return on Investment (SROI) methodology to account for value for money of public health interventions: a systematic review. BMC Public Health. 2015;15:582.CrossRefPubMedPubMedCentral
46.
go back to reference Caro JJ, Briggs AH, Siebert U, Kuntz KM, Force I-SMGRPT. Modeling good research practices--overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--1. Value Health. 2012;15(6):796–803.CrossRefPubMed Caro JJ, Briggs AH, Siebert U, Kuntz KM, Force I-SMGRPT. Modeling good research practices--overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--1. Value Health. 2012;15(6):796–803.CrossRefPubMed
47.
go back to reference Cookson R, Drummond M, Weatherly H. Explicit incorporation of equity considerations into economic evaluation of public health interventions. Health Econ Policy Law. 2009;4(Pt 2):231–45.CrossRefPubMed Cookson R, Drummond M, Weatherly H. Explicit incorporation of equity considerations into economic evaluation of public health interventions. Health Econ Policy Law. 2009;4(Pt 2):231–45.CrossRefPubMed
48.
go back to reference Drost RM, Paulus AT, Ruwaard D, Evers SM. Inter-sectoral costs and benefits of mental health prevention: towards a new classification scheme. J Ment Health Policy Econ. 2013;16(4):179–86.PubMed Drost RM, Paulus AT, Ruwaard D, Evers SM. Inter-sectoral costs and benefits of mental health prevention: towards a new classification scheme. J Ment Health Policy Econ. 2013;16(4):179–86.PubMed
Metadata
Title
A qualitative study on the views of experts regarding the incorporation of non-health outcomes into the economic evaluations of public health interventions
Authors
Ghislaine APG van Mastrigt
Aggie TG Paulus
Marie-Jeanne Aarts
Silvia MAA Evers
Adrienne FG Alayli-Goebbels
Publication date
01-12-2015
Publisher
BioMed Central
Published in
BMC Public Health / Issue 1/2015
Electronic ISSN: 1471-2458
DOI
https://doi.org/10.1186/s12889-015-2247-7

Other articles of this Issue 1/2015

BMC Public Health 1/2015 Go to the issue