Skip to main content
Top
Published in: PharmacoEconomics 10/2007

01-10-2007 | Practical Application

Lessons from Trial-Based Cost-Effectiveness Analyses of Mental Health Interventions

Why Uncertainty About the Outcome, Estimate and Willingness to Pay Matters

Authors: Dr Jeffrey S. Hoch, Carolyn S. Dewa

Published in: PharmacoEconomics | Issue 10/2007

Login to get access

Abstract

The principal aim of this article is to share lessons learned by the authors while conducting economic evaluations, using clinical trial data, of mental health interventions. These lessons are quite general and have clear relevance for pharmacoeconomic studies. In addition, we explore how net benefit regression can be used to enhance consideration of key issues when conducting an economic evaluation based on clinical trial data.
The first study we discuss found that cost-effectiveness results varied markedly based on the choice of both the patient outcome and the willingness to pay for more of that outcome. The importance of willingness to pay was also highlighted in the results from the second study. Even with a set willingness-to-pay value, most of the time the probability that the new treatment was cost effective was not 100%. In the third study, the cost effectiveness of the new treatment varied by patient characteristics.
These observations have important implications for pharmacoeconomic studies. Namely, analysts must carefully consider choice of patient outcome, willingness to pay, patient heterogeneity and the statistical uncertainty inherent in the data. Net benefit regression is a useful technique for exploring these crucial issues when undertaking an economic evaluation using patient-level data on both costs and effects.
Appendix
Available only for authorised users
Footnotes
1
When non-parametric methods are desired, bootstrapping can be used to make a confidence interval for the estimate of β1.
 
Literature
1.
go back to reference Evers S, Van Wijk A, Ament A. Economic evaluation of mental health care interventions: a review. Health Econ 1997; 6: 161–177PubMedCrossRef Evers S, Van Wijk A, Ament A. Economic evaluation of mental health care interventions: a review. Health Econ 1997; 6: 161–177PubMedCrossRef
2.
go back to reference Drummond M. Method without madness. Expert Rev Pharmacoeconomics Outcomes Res 2002; 2: 305–307CrossRef Drummond M. Method without madness. Expert Rev Pharmacoeconomics Outcomes Res 2002; 2: 305–307CrossRef
3.
go back to reference Dixon L, Hoch J, Clark R, et al. Cost-effectiveness of two vocational rehabilitation programs for persons with severe mental illness. Psychiatric Services 2002; 53: 1118–1124PubMedCrossRef Dixon L, Hoch J, Clark R, et al. Cost-effectiveness of two vocational rehabilitation programs for persons with severe mental illness. Psychiatric Services 2002; 53: 1118–1124PubMedCrossRef
4.
go back to reference Fenton W, Hoch J, Herrell J, et al. Cost and cost-effectiveness of hospital vs residential crisis care for patients who have serious mental illness. Arch Gen Psychiatry 2002; 59: 357–364PubMedCrossRef Fenton W, Hoch J, Herrell J, et al. Cost and cost-effectiveness of hospital vs residential crisis care for patients who have serious mental illness. Arch Gen Psychiatry 2002; 59: 357–364PubMedCrossRef
5.
go back to reference Lehman A, Dixon L, Hoch J. Cost-effectiveness of assertive community treatment for homeless persons with severe mental illness. Br J Psychiatry 1999; 174: 346–352PubMedCrossRef Lehman A, Dixon L, Hoch J. Cost-effectiveness of assertive community treatment for homeless persons with severe mental illness. Br J Psychiatry 1999; 174: 346–352PubMedCrossRef
6.
go back to reference Pritchard C. Trends in economic evaluation [OHE brief]. London: Office of Health Economics, 1998: 36 Pritchard C. Trends in economic evaluation [OHE brief]. London: Office of Health Economics, 1998: 36
7.
go back to reference Briggs A, O’Brien B, Blackhouse G. Thinking outside the box: recent advances in the analysis and presentation of uncertainty in cost-effectiveness studies. Annu Rev Public Health 2002; 23: 377–401PubMedCrossRef Briggs A, O’Brien B, Blackhouse G. Thinking outside the box: recent advances in the analysis and presentation of uncertainty in cost-effectiveness studies. Annu Rev Public Health 2002; 23: 377–401PubMedCrossRef
8.
go back to reference Hoch J, Briggs A, Willan A. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ 2002; 11 (5): 415–430PubMedCrossRef Hoch J, Briggs A, Willan A. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ 2002; 11 (5): 415–430PubMedCrossRef
9.
go back to reference Bond G, Drake R, Mueser K, et al. An update on supported employment for people with severe mental illness. Psychiatr Serv 1997; 48: 335–346PubMed Bond G, Drake R, Mueser K, et al. An update on supported employment for people with severe mental illness. Psychiatr Serv 1997; 48: 335–346PubMed
10.
go back to reference Lehman A, Goldberg R, Dixon L, et al. Improving employment outcomes for persons with severe mental illness. Arch Gen Psychiatry 2002; 59: 165–172PubMedCrossRef Lehman A, Goldberg R, Dixon L, et al. Improving employment outcomes for persons with severe mental illness. Arch Gen Psychiatry 2002; 59: 165–172PubMedCrossRef
11.
go back to reference Clark R, Bond G. Costs and benefits of vocational programs for people with serious mental illness. In: Moscarelli M, Rupp A, Sartorius N, editors. Handbook of mental health economics and health policy. Vol. 1. Sussex: John Wiley& Sons, Ltd, 1996: 219–237 Clark R, Bond G. Costs and benefits of vocational programs for people with serious mental illness. In: Moscarelli M, Rupp A, Sartorius N, editors. Handbook of mental health economics and health policy. Vol. 1. Sussex: John Wiley& Sons, Ltd, 1996: 219–237
12.
go back to reference Drake R, McHugo G, Bebout R, et al. A randomized clinical trial of supported employment for innercity patients with severe mental disorders. Arch Gen Psychiatry 1999; 56: 627–633PubMedCrossRef Drake R, McHugo G, Bebout R, et al. A randomized clinical trial of supported employment for innercity patients with severe mental disorders. Arch Gen Psychiatry 1999; 56: 627–633PubMedCrossRef
13.
go back to reference Stroul B. Crisis residential services in a community support system. Rockville (MD): National Institute of Mental Health, 1987 Stroul B. Crisis residential services in a community support system. Rockville (MD): National Institute of Mental Health, 1987
14.
go back to reference Fenton W, Mosher L, Herell J, et al. Randomized trial of general hospital and residential alternative care for patients with severe and persistent mental illness. Am J Psychiatry 1998; 155: 516–522PubMed Fenton W, Mosher L, Herell J, et al. Randomized trial of general hospital and residential alternative care for patients with severe and persistent mental illness. Am J Psychiatry 1998; 155: 516–522PubMed
15.
go back to reference Fenton W, Mosher L. Crisis residential care for patients with serious mental illness. In: Martindale BV, Batemean A, Crowe M, editors. Psychosis: psychological approaches and their effectiveness. London: Gaskell Press, 2000 Fenton W, Mosher L. Crisis residential care for patients with serious mental illness. In: Martindale BV, Batemean A, Crowe M, editors. Psychosis: psychological approaches and their effectiveness. London: Gaskell Press, 2000
16.
go back to reference O’Brien B, Sculpher M. Building uncertainty into cost-effectiveness rankings: portfolio risk-return tradeoffs and implications for decision rules. Med Care 2000; 38: 460–468PubMedCrossRef O’Brien B, Sculpher M. Building uncertainty into cost-effectiveness rankings: portfolio risk-return tradeoffs and implications for decision rules. Med Care 2000; 38: 460–468PubMedCrossRef
17.
go back to reference Chaudhary M, Stearns S. Estimating confidence intervals for cost-effectiveness ratios: an example from a randomized trial. Stat Med 1996; 15: 1447–1458PubMedCrossRef Chaudhary M, Stearns S. Estimating confidence intervals for cost-effectiveness ratios: an example from a randomized trial. Stat Med 1996; 15: 1447–1458PubMedCrossRef
18.
go back to reference Stein L, Test M. Alternative to mental hospital treatment: I. Conceptual model, treatment program, and clinical evaluation. Arch Gen Psychiatry 1980; 37: 392–397PubMedCrossRef Stein L, Test M. Alternative to mental hospital treatment: I. Conceptual model, treatment program, and clinical evaluation. Arch Gen Psychiatry 1980; 37: 392–397PubMedCrossRef
19.
go back to reference Olfson M. Assertive community treatment: an evaluation of the experimental evidence. Hosp Commun Psychiatry 1990; 41: 634–641 Olfson M. Assertive community treatment: an evaluation of the experimental evidence. Hosp Commun Psychiatry 1990; 41: 634–641
20.
go back to reference Burns B, Santos A. Assertive community treatment: an update of randomized trials. Psychiatr Serv 1995; 46: 669–675PubMed Burns B, Santos A. Assertive community treatment: an update of randomized trials. Psychiatr Serv 1995; 46: 669–675PubMed
21.
go back to reference Scott J, Dixon L. Assertive community treatment and case management for schizophrenia. Schizophrenia Bull 1995; 21: 657–668CrossRef Scott J, Dixon L. Assertive community treatment and case management for schizophrenia. Schizophrenia Bull 1995; 21: 657–668CrossRef
22.
go back to reference Lehman A, Dixon L, Kernan E, et al. A randomized trial of assertive community treatment for homeless persons with severe mental illness. Arch Gen Psychiatry 1997; 54: 1038–1043PubMedCrossRef Lehman A, Dixon L, Kernan E, et al. A randomized trial of assertive community treatment for homeless persons with severe mental illness. Arch Gen Psychiatry 1997; 54: 1038–1043PubMedCrossRef
23.
go back to reference McDonough CM, Tosteson AN. Measuring preferences for cost-utility analysis: how choice of method may influence decision-making. Pharmacoeconomics 2007; 25 (2): 93–106PubMedCrossRef McDonough CM, Tosteson AN. Measuring preferences for cost-utility analysis: how choice of method may influence decision-making. Pharmacoeconomics 2007; 25 (2): 93–106PubMedCrossRef
24.
go back to reference Gold M, Stevenson D, Fryback D. HALYS and QALYS and D ALYS, oh my: similarities and differences in summary measures of population Health. Annu Rev Public Health 2002; 23: 115–134PubMedCrossRef Gold M, Stevenson D, Fryback D. HALYS and QALYS and D ALYS, oh my: similarities and differences in summary measures of population Health. Annu Rev Public Health 2002; 23: 115–134PubMedCrossRef
25.
go back to reference Krabbe P, Essink-Bot M, Bonsel G. The comparability and reliability of five health-state valuation methods. Soc Sci Medicine 1997; 45: 1641–1652CrossRef Krabbe P, Essink-Bot M, Bonsel G. The comparability and reliability of five health-state valuation methods. Soc Sci Medicine 1997; 45: 1641–1652CrossRef
26.
go back to reference Arnesen T, Trommald M. Roughly right or precisely wrong? Systematic review of quality-of-life weights elicited with the time trade-off method. J Health Serv Res Policy 2004; 9: 43–50PubMedCrossRef Arnesen T, Trommald M. Roughly right or precisely wrong? Systematic review of quality-of-life weights elicited with the time trade-off method. J Health Serv Res Policy 2004; 9: 43–50PubMedCrossRef
27.
go back to reference Lamers L, Bouwmans C, Straten A, et al. Comparison of EQ-5D and SF-6D utilities in mental health patients. Health Econ 2006; 15: 1229–1236PubMedCrossRef Lamers L, Bouwmans C, Straten A, et al. Comparison of EQ-5D and SF-6D utilities in mental health patients. Health Econ 2006; 15: 1229–1236PubMedCrossRef
28.
go back to reference Shumway M. Preference weights for cost-outcome analyses of schizophrenia treatments: comparison of four stakeholder groups. Schizophr Bull 2003; 29: 257–266PubMedCrossRef Shumway M. Preference weights for cost-outcome analyses of schizophrenia treatments: comparison of four stakeholder groups. Schizophr Bull 2003; 29: 257–266PubMedCrossRef
29.
go back to reference McGregor M, Caro J. QALYs: are they helpful to decision makers? Pharmacoeconomics Outcomes Res 2006; 24: 947–952 McGregor M, Caro J. QALYs: are they helpful to decision makers? Pharmacoeconomics Outcomes Res 2006; 24: 947–952
30.
go back to reference Hoch J. Using kernel density estimators to illustrate why cost-effectiveness varies by patient subgroups: promoting the kernel to general (use) [poster]. Annual Meeting of the Society for Medical Decision Making; 2005 Oct 21–24; San Francisco Hoch J. Using kernel density estimators to illustrate why cost-effectiveness varies by patient subgroups: promoting the kernel to general (use) [poster]. Annual Meeting of the Society for Medical Decision Making; 2005 Oct 21–24; San Francisco
31.
go back to reference Hoch J. Cost-effectiveness lessons from disease-modifying drugs in the treatment of multiple sclerosis. Exp Rev Pharmacoeconomics Outcomes Res 2004; 4: 537–547CrossRef Hoch J. Cost-effectiveness lessons from disease-modifying drugs in the treatment of multiple sclerosis. Exp Rev Pharmacoeconomics Outcomes Res 2004; 4: 537–547CrossRef
32.
go back to reference Hoch J, Dewa C. An introduction to economic evaluation: what’s in a name? Can J Psychiatry 2005; 50: 159–166PubMed Hoch J, Dewa C. An introduction to economic evaluation: what’s in a name? Can J Psychiatry 2005; 50: 159–166PubMed
33.
go back to reference Stinnett A, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decision Making 1998; 18 (2 Suppl.): S68–S80CrossRef Stinnett A, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decision Making 1998; 18 (2 Suppl.): S68–S80CrossRef
34.
go back to reference Tambour M, Zethraeus N, Johannesson M. A note on confidence intervals in cost-effectiveness analysis. Int J Technol Assess Health Care 1998; 14: 467–471PubMedCrossRef Tambour M, Zethraeus N, Johannesson M. A note on confidence intervals in cost-effectiveness analysis. Int J Technol Assess Health Care 1998; 14: 467–471PubMedCrossRef
35.
go back to reference Hoch JS, Smith MW. A guide to economic evaluation: methods for cost-effectiveness analysis of person-level data. J Trauma Stress 2006 Dec; 19 (6): 787–797PubMedCrossRef Hoch JS, Smith MW. A guide to economic evaluation: methods for cost-effectiveness analysis of person-level data. J Trauma Stress 2006 Dec; 19 (6): 787–797PubMedCrossRef
36.
go back to reference Kopans DB. An overview of the breast cancer screening controversy. J Natl Cancer Inst Monogr 1997; 22: 1–3PubMed Kopans DB. An overview of the breast cancer screening controversy. J Natl Cancer Inst Monogr 1997; 22: 1–3PubMed
37.
go back to reference van Hout B, Al M, Gordon G, et al. Costs, effects and C/E-ratios alongside a clinical trial. Health Econ 1994; 3: 309–319PubMedCrossRef van Hout B, Al M, Gordon G, et al. Costs, effects and C/E-ratios alongside a clinical trial. Health Econ 1994; 3: 309–319PubMedCrossRef
38.
go back to reference Fenwick E Claxton K, Sculpher M. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Econ 2001; 10: 779–787PubMedCrossRef Fenwick E Claxton K, Sculpher M. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Econ 2001; 10: 779–787PubMedCrossRef
39.
go back to reference Fenwick E, Byford S. A guide to cost-effectiveness acceptability curves. Br J Psychiatry 2005; 187: 106–108PubMedCrossRef Fenwick E, Byford S. A guide to cost-effectiveness acceptability curves. Br J Psychiatry 2005; 187: 106–108PubMedCrossRef
40.
go back to reference Lothgren M, Zethraeus N. Definition, interpretation and calculation of cost-effectiveness acceptability curves. Health Econ 2000: 9: 623–630PubMedCrossRef Lothgren M, Zethraeus N. Definition, interpretation and calculation of cost-effectiveness acceptability curves. Health Econ 2000: 9: 623–630PubMedCrossRef
41.
go back to reference Fenwick E, Marshall D, Levy A, et al. Using and interpreting cost-effectiveness acceptability curves: an example using data from a trial of management strategies for atrial fibrillation. BMC Health Serv Res 2006; 6: 52PubMedCrossRef Fenwick E, Marshall D, Levy A, et al. Using and interpreting cost-effectiveness acceptability curves: an example using data from a trial of management strategies for atrial fibrillation. BMC Health Serv Res 2006; 6: 52PubMedCrossRef
42.
go back to reference Hoch J, Rockx M, Krahn A. Using the net benefit regression framework to construct cost-effectiveness acceptability curves: an example using data from a trial of external loop recorders versus Holter monitoring for ambulatory monitoring of’ community acquired’ syncope. BMC Health Serv Res 2006 Jun; 6: 68PubMedCrossRef Hoch J, Rockx M, Krahn A. Using the net benefit regression framework to construct cost-effectiveness acceptability curves: an example using data from a trial of external loop recorders versus Holter monitoring for ambulatory monitoring of’ community acquired’ syncope. BMC Health Serv Res 2006 Jun; 6: 68PubMedCrossRef
43.
go back to reference Zethraeus N, Johannesson M, Jonsson B, et al. Advantages of using the net-benefit approach for analysing uncertainty in economic evaluation studies. Pharmacoeconomics Outcomes Res 2003; 21: 39–48 Zethraeus N, Johannesson M, Jonsson B, et al. Advantages of using the net-benefit approach for analysing uncertainty in economic evaluation studies. Pharmacoeconomics Outcomes Res 2003; 21: 39–48
44.
go back to reference Drummond MF, McGuire A. Economic evaluation in health care: merging theory with practice. New York: Oxford University Press, 2001 Drummond MF, McGuire A. Economic evaluation in health care: merging theory with practice. New York: Oxford University Press, 2001
45.
go back to reference Drummond M, Sculpher M, Torrance G, et al. Methods for the economic evaluation of health care programmes. New York: Oxford University Press, 2005 Drummond M, Sculpher M, Torrance G, et al. Methods for the economic evaluation of health care programmes. New York: Oxford University Press, 2005
46.
go back to reference Ramsey S, Willke R, Briggs A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA task force report. Value Health 2005; 8: 521–533PubMedCrossRef Ramsey S, Willke R, Briggs A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA task force report. Value Health 2005; 8: 521–533PubMedCrossRef
47.
go back to reference Doshi J, Glick H, Polsky D. Analyses of cost data in economic evaluations conducted alongside randomized controlled trials. Value Health 2006; 9: 334–340PubMedCrossRef Doshi J, Glick H, Polsky D. Analyses of cost data in economic evaluations conducted alongside randomized controlled trials. Value Health 2006; 9: 334–340PubMedCrossRef
48.
go back to reference Polsky D, Doshi J, Bauer M, et al. Clinical trial-based cost-effectiveness analyses of antipsychotic use. Am J Psychiatry 2006; 163: 2047–2056PubMedCrossRef Polsky D, Doshi J, Bauer M, et al. Clinical trial-based cost-effectiveness analyses of antipsychotic use. Am J Psychiatry 2006; 163: 2047–2056PubMedCrossRef
49.
go back to reference Manca A, Palmer S. Handling missing data in patient-level cost-effectiveness analysis alongside randomised clinical trials. Appl Health Econ Health Policy 2005; 4: 65–75PubMedCrossRef Manca A, Palmer S. Handling missing data in patient-level cost-effectiveness analysis alongside randomised clinical trials. Appl Health Econ Health Policy 2005; 4: 65–75PubMedCrossRef
50.
go back to reference Briggs A, Nixon R, Dixon S, et al. Parametric modelling of cost data: some simulation evidence. Health Econ 2005; 14: 421–428PubMedCrossRef Briggs A, Nixon R, Dixon S, et al. Parametric modelling of cost data: some simulation evidence. Health Econ 2005; 14: 421–428PubMedCrossRef
51.
go back to reference Basu A, Arondekar B, Rathouz P. Scale of interest versus scale of estimation: comparing alternative estimators for the incremental costs of a comorbidity. Health Econ 2006; 15: 1091–1107PubMedCrossRef Basu A, Arondekar B, Rathouz P. Scale of interest versus scale of estimation: comparing alternative estimators for the incremental costs of a comorbidity. Health Econ 2006; 15: 1091–1107PubMedCrossRef
52.
go back to reference Nixon R, Thompson S. Methods for incorporating covariate adjustment, subgroup analysis and between-centre differences into cost-effectiveness evaluations. Health Econ 2005; 14: 1217–1229PubMedCrossRef Nixon R, Thompson S. Methods for incorporating covariate adjustment, subgroup analysis and between-centre differences into cost-effectiveness evaluations. Health Econ 2005; 14: 1217–1229PubMedCrossRef
53.
go back to reference Manca A, Rice N, Sculpher M, et al. Assessing generalisability by location in trial-based cost-effectiveness analysis: the use of multilevel models. Health Econ 2005; 14: 471–485PubMedCrossRef Manca A, Rice N, Sculpher M, et al. Assessing generalisability by location in trial-based cost-effectiveness analysis: the use of multilevel models. Health Econ 2005; 14: 471–485PubMedCrossRef
54.
go back to reference Willan A, Briggs A, Hoch J. Regression methods for covariate adjustment and subgroup analysis for non-censored cost-effectiveness data. Health Econ 2004; 13: 461–475PubMedCrossRef Willan A, Briggs A, Hoch J. Regression methods for covariate adjustment and subgroup analysis for non-censored cost-effectiveness data. Health Econ 2004; 13: 461–475PubMedCrossRef
55.
go back to reference Mahoney E, Mehta S, Yuan Y, et al. Long-term cost-effectiveness of early and sustained clopidogrel therapy for up to 1 year in patients undergoing percutaneous coronary intervention after presenting with acute coronary syndromes without ST-segment elevation. Am Heart J 2006; 151: 219–227PubMedCrossRef Mahoney E, Mehta S, Yuan Y, et al. Long-term cost-effectiveness of early and sustained clopidogrel therapy for up to 1 year in patients undergoing percutaneous coronary intervention after presenting with acute coronary syndromes without ST-segment elevation. Am Heart J 2006; 151: 219–227PubMedCrossRef
Metadata
Title
Lessons from Trial-Based Cost-Effectiveness Analyses of Mental Health Interventions
Why Uncertainty About the Outcome, Estimate and Willingness to Pay Matters
Authors
Dr Jeffrey S. Hoch
Carolyn S. Dewa
Publication date
01-10-2007
Publisher
Springer International Publishing
Published in
PharmacoEconomics / Issue 10/2007
Print ISSN: 1170-7690
Electronic ISSN: 1179-2027
DOI
https://doi.org/10.2165/00019053-200725100-00001

Other articles of this Issue 10/2007

PharmacoEconomics 10/2007 Go to the issue