Skip to main content
Top
Published in: The European Journal of Health Economics 1/2018

Open Access 01-01-2018 | Original Paper

A methodological framework for assessing agreement between cost-effectiveness outcomes estimated using alternative sources of data on treatment costs and effects for trial-based economic evaluations

Authors: Felix Achana, Stavros Petrou, Kamran Khan, Amadou Gaye, Neena Modi, On behalf of Medicines for Neonates Investigators

Published in: The European Journal of Health Economics | Issue 1/2018

Login to get access

Abstract

A new methodological framework for assessing agreement between cost-effectiveness endpoints generated using alternative sources of data on treatment costs and effects for trial-based economic evaluations is proposed. The framework can be used to validate cost-effectiveness endpoints generated from routine data sources when comparable data is available directly from trial case report forms or from another source. We illustrate application of the framework using data from a recent trial-based economic evaluation of the probiotic Bifidobacterium breve strain BBG administered to babies less than 31 weeks of gestation. Cost-effectiveness endpoints are compared using two sources of information; trial case report forms and data extracted from the National Neonatal Research Database (NNRD), a clinical database created through collaborative efforts of UK neonatal services. Focusing on mean incremental net benefits at £30,000 per episode of sepsis averted, the study revealed no evidence of discrepancy between the data sources (two-sided p values >0.4), low probability estimates of miscoverage (ranging from 0.039 to 0.060) and concordance correlation coefficients greater than 0.86. We conclude that the NNRD could potentially serve as a reliable source of data for future trial-based economic evaluations of neonatal interventions. We also discuss the potential implications of increasing opportunity to utilize routinely available data for the conduct of trial-based economic evaluations.
Appendix
Available only for authorised users
Literature
1.
go back to reference Altman, D.G., Bland, J.M.: Measurement in medicine: the analysis of method comparison studies. J. R. Stat. Soc. Ser. D (Stat.) 32(3), 307–317 (1983) Altman, D.G., Bland, J.M.: Measurement in medicine: the analysis of method comparison studies. J. R. Stat. Soc. Ser. D (Stat.) 32(3), 307–317 (1983)
2.
go back to reference Bell, M.J., Ternberg, J.L., Feigin, R.D., Keating, J.P., Marshall, R., Barton, L., Brotherton, T.: Neonatal necrotizing enterocolitis. Therapeutic decisions based upon clinical staging. Ann. Surg. 187(1), 1–7 (1978)CrossRef Bell, M.J., Ternberg, J.L., Feigin, R.D., Keating, J.P., Marshall, R., Barton, L., Brotherton, T.: Neonatal necrotizing enterocolitis. Therapeutic decisions based upon clinical staging. Ann. Surg. 187(1), 1–7 (1978)CrossRef
3.
go back to reference Bland, J.M., Altman, D.G.: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 327(8476), 307–310 (1986)CrossRef Bland, J.M., Altman, D.G.: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 327(8476), 307–310 (1986)CrossRef
4.
go back to reference Byford, S., Leese, M., Knapp, M., Seivewright, H., Cameron, S., Jones, V., Davidson, K., Tyrer, P.: Comparison of alternative methods of collection of service use data for the economic evaluation of health care interventions. Health Econ. 16(5), 531–536 (2007)CrossRef Byford, S., Leese, M., Knapp, M., Seivewright, H., Cameron, S., Jones, V., Davidson, K., Tyrer, P.: Comparison of alternative methods of collection of service use data for the economic evaluation of health care interventions. Health Econ. 16(5), 531–536 (2007)CrossRef
5.
go back to reference Carrasco, J.L., King, T.S., Chinchilli, V.M.: The concordance correlation coefficient for repeated measures estimated by variance components. J. Biopharm. Stat. 19(1), 90–105 (2009)CrossRef Carrasco, J.L., King, T.S., Chinchilli, V.M.: The concordance correlation coefficient for repeated measures estimated by variance components. J. Biopharm. Stat. 19(1), 90–105 (2009)CrossRef
6.
go back to reference Costeloe, K., Hardy, P., Juszczak, E., Wilks, M., Millar, M.R.: Bifidobacterium breve BBG-001 in very preterm infants: a randomised controlled phase 3 trial. Lancet 387(10019), 649–660 (2016)CrossRef Costeloe, K., Hardy, P., Juszczak, E., Wilks, M., Millar, M.R.: Bifidobacterium breve BBG-001 in very preterm infants: a randomised controlled phase 3 trial. Lancet 387(10019), 649–660 (2016)CrossRef
7.
go back to reference Costeloe, K., Wilks, M., Hardy, P., Nelis, C., Millar, M.: O-008 Early bifidobacterium breve Bbg-001 to prevent necrotising enterocolitis, late-onset sepsis and death: the PiPS Trial. Arch. Dis. Child. 99(Suppl 2), A23–A24 (2014)CrossRef Costeloe, K., Wilks, M., Hardy, P., Nelis, C., Millar, M.: O-008 Early bifidobacterium breve Bbg-001 to prevent necrotising enterocolitis, late-onset sepsis and death: the PiPS Trial. Arch. Dis. Child. 99(Suppl 2), A23–A24 (2014)CrossRef
10.
go back to reference Dowie, J.: Why cost-effectiveness should trump (clinical) effectiveness: the ethical economics of the South West quadrant. Health Econ. 13(5), 453–459 (2004)CrossRef Dowie, J.: Why cost-effectiveness should trump (clinical) effectiveness: the ethical economics of the South West quadrant. Health Econ. 13(5), 453–459 (2004)CrossRef
11.
go back to reference Feng, D., Baumgartner, R., Svetnik, V.: A robust Bayesian estimate of the concordance correlation coefficient. J. Biopharm. Stat. 25(3), 490–507 (2014)CrossRef Feng, D., Baumgartner, R., Svetnik, V.: A robust Bayesian estimate of the concordance correlation coefficient. J. Biopharm. Stat. 25(3), 490–507 (2014)CrossRef
12.
go back to reference Glick, H.A., Doshi, J.A., Sonnad, S.S., Polsky, D.: Economic Evaluation in Clinical Trials. Oxford University Press, Oxford (2015) Glick, H.A., Doshi, J.A., Sonnad, S.S., Polsky, D.: Economic Evaluation in Clinical Trials. Oxford University Press, Oxford (2015)
13.
go back to reference Grieve, R., Abrams, K., Claxton, K., Goldacre, B., James, N., Nicholl, J., Parmar, M., Parker, C., Sekhon, J.S., Smeeth, L., Spiegelhalter, D., Sculpher M.: Cancer Drugs Fund requires further reform reliance on “real world” observational data undermines evidence base for clinical practice. Br. Med. J. 354 (2016) Grieve, R., Abrams, K., Claxton, K., Goldacre, B., James, N., Nicholl, J., Parmar, M., Parker, C., Sekhon, J.S., Smeeth, L., Spiegelhalter, D., Sculpher M.: Cancer Drugs Fund requires further reform reliance on “real world” observational data undermines evidence base for clinical practice. Br. Med. J. 354 (2016)
14.
go back to reference Hiriote, S., Chinchilli, V.M.: Matrix-based concordance correlation coefficient for repeated measures. Biometrics 67(3), 1007–1016 (2011)CrossRef Hiriote, S., Chinchilli, V.M.: Matrix-based concordance correlation coefficient for repeated measures. Biometrics 67(3), 1007–1016 (2011)CrossRef
15.
go back to reference Houweling, T., Bolton, J., Newell, D.: Comparison of two methods of collecting healthcare usage data in chiropractic clinics: patient-report versus documentation in patient files. Chiropr. Man. Ther. 22, 32 (2014)CrossRef Houweling, T., Bolton, J., Newell, D.: Comparison of two methods of collecting healthcare usage data in chiropractic clinics: patient-report versus documentation in patient files. Chiropr. Man. Ther. 22, 32 (2014)CrossRef
16.
go back to reference King, T.S., Chinchilli, V.M., Carrasco, J.L.: A repeated measures concordance correlation coefficient. Stat. Med. 26(16), 3095–3113 (2007)CrossRef King, T.S., Chinchilli, V.M., Carrasco, J.L.: A repeated measures concordance correlation coefficient. Stat. Med. 26(16), 3095–3113 (2007)CrossRef
17.
go back to reference Li, R., Chow, M.: Evaluation of reproducibility for paired functional data. J. Multivar. Anal. 93(1), 81–101 (2005)CrossRef Li, R., Chow, M.: Evaluation of reproducibility for paired functional data. J. Multivar. Anal. 93(1), 81–101 (2005)CrossRef
18.
go back to reference Lin, L.I.K.: A concordance correlation coefficient to evaluate reproducibility. Biometrics 45(1), 255–268 (1989)CrossRef Lin, L.I.K.: A concordance correlation coefficient to evaluate reproducibility. Biometrics 45(1), 255–268 (1989)CrossRef
19.
go back to reference Lin, L.I.K.: Assay validation using the concordance correlation coefficient. Biometrics 48(2), 599–604 (1992)CrossRef Lin, L.I.K.: Assay validation using the concordance correlation coefficient. Biometrics 48(2), 599–604 (1992)CrossRef
20.
go back to reference Lin, L.I.K.: Correction: a note on the concordance correlation coefficient. Biometrics 56(1), 324–325 (2000)CrossRef Lin, L.I.K.: Correction: a note on the concordance correlation coefficient. Biometrics 56(1), 324–325 (2000)CrossRef
21.
go back to reference Mant, J., Murphy, M., Rose, P., Vessey, M.: The accuracy of general practitioner records of smoking and alcohol use: comparison with patient questionnaires. J. Public Health 22(2), 198–201 (2000)CrossRef Mant, J., Murphy, M., Rose, P., Vessey, M.: The accuracy of general practitioner records of smoking and alcohol use: comparison with patient questionnaires. J. Public Health 22(2), 198–201 (2000)CrossRef
22.
go back to reference McBride, G.B.: A Proposal for strength-of-agreement criteria for Lin’s concordance correlation coefficient. NIWA Client Report: HAM2005-062. Report to Ministry of Health (2005) McBride, G.B.: A Proposal for strength-of-agreement criteria for Lin’s concordance correlation coefficient. NIWA Client Report: HAM2005-062. Report to Ministry of Health (2005)
23.
go back to reference Mistry, H., Buxton, M., Longworth, L., Chatwin, J., Peveler, R.: Comparison of general practitioner records and patient self-report questionnaires for estimation of costs. Eur. J. Health Econ. 6(3), 261–266 (2005)CrossRef Mistry, H., Buxton, M., Longworth, L., Chatwin, J., Peveler, R.: Comparison of general practitioner records and patient self-report questionnaires for estimation of costs. Eur. J. Health Econ. 6(3), 261–266 (2005)CrossRef
24.
go back to reference National Institute for Health and Care Excellence, (NICE). (2013). Guide to the methods of technology appraisal. National Institute for Health and Care Excellence (NICE), London National Institute for Health and Care Excellence, (NICE). (2013). Guide to the methods of technology appraisal. National Institute for Health and Care Excellence (NICE), London
25.
go back to reference O’Brien, B.J., Gertsen, K., Willan, A.R., Faulkner, A.: Is there a kink in consumers’ threshold value for cost-effectiveness in health care? Health Econ. 11(2), 175–180 (2002)CrossRef O’Brien, B.J., Gertsen, K., Willan, A.R., Faulkner, A.: Is there a kink in consumers’ threshold value for cost-effectiveness in health care? Health Econ. 11(2), 175–180 (2002)CrossRef
26.
go back to reference Petrou, S., Gray A.: Economic evaluation alongside randomised controlled trials: design, conduct, analysis, and reporting. BMJ 342 (2011) Petrou, S., Gray A.: Economic evaluation alongside randomised controlled trials: design, conduct, analysis, and reporting. BMJ 342 (2011)
27.
go back to reference Polsky, D., Glick, H.A., Willke, R., Schulman, K.: Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Econ. 6(3), 243–252 (1997)CrossRef Polsky, D., Glick, H.A., Willke, R., Schulman, K.: Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Econ. 6(3), 243–252 (1997)CrossRef
29.
go back to reference Raftery, J., Roderick, P., Stevens, A.: Potential use of routine databases in health technology assessment. Health Technol. Assess. 9(20), 1–92, iii–iv (2005)CrossRef Raftery, J., Roderick, P., Stevens, A.: Potential use of routine databases in health technology assessment. Health Technol. Assess. 9(20), 1–92, iii–iv (2005)CrossRef
30.
go back to reference Schroeder, E., Petrou, S., Patel, N., Hollowell, J., Puddicombe, D., Redshaw, M., Brocklehurst, P.: Cost effectiveness of alternative planned places of birth in woman at low risk of complications: evidence from the Birthplace in England national prospective cohort study. BMJ 344 (2012) Schroeder, E., Petrou, S., Patel, N., Hollowell, J., Puddicombe, D., Redshaw, M., Brocklehurst, P.: Cost effectiveness of alternative planned places of birth in woman at low risk of complications: evidence from the Birthplace in England national prospective cohort study. BMJ 344 (2012)
31.
go back to reference Smith, W., McCrone, P., Goddard, C., Gao, W., Burman, R., Jackson, D., Higginson, I., Silber, E., Koffman, J.: Comparisons of costs between black Caribbean and white British patients with advanced multiple sclerosis in the UK. Mult. Scler. Int. 2014, 613701 (2014)PubMedPubMedCentral Smith, W., McCrone, P., Goddard, C., Gao, W., Burman, R., Jackson, D., Higginson, I., Silber, E., Koffman, J.: Comparisons of costs between black Caribbean and white British patients with advanced multiple sclerosis in the UK. Mult. Scler. Int. 2014, 613701 (2014)PubMedPubMedCentral
32.
go back to reference StataCorp.: Stata statistical software: release 12. StataCorp LP, College Station (2011) StataCorp.: Stata statistical software: release 12. StataCorp LP, College Station (2011)
33.
go back to reference Stinnett, A.A., Mullahy, J.: Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med. Decis. Making 18(2), S68–S80 (1998)CrossRef Stinnett, A.A., Mullahy, J.: Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med. Decis. Making 18(2), S68–S80 (1998)CrossRef
35.
go back to reference Thompson, S.G., Barber, J.A.: How should cost data in pragmatic randomised trials be analysed? BMJ Br. Med. J. 320(7243), 1197 (2000)CrossRef Thompson, S.G., Barber, J.A.: How should cost data in pragmatic randomised trials be analysed? BMJ Br. Med. J. 320(7243), 1197 (2000)CrossRef
36.
go back to reference Thorn, J.C., Turner, E.L., Hounsome, L., Walsh, E., Down, L., Verne, J., Donovan, J.L., Neal, D.E., Hamdy, F.C., Martin, R.M., Noble, S.M., C. A. P. t. group: Validating the use of Hospital Episode Statistics data and comparison of costing methodologies for economic evaluation: an end-of-life case study from the Cluster randomised triAl of PSA testing for Prostate cancer (CAP). BMJ Open 6(4), e011063 (2016)CrossRef Thorn, J.C., Turner, E.L., Hounsome, L., Walsh, E., Down, L., Verne, J., Donovan, J.L., Neal, D.E., Hamdy, F.C., Martin, R.M., Noble, S.M., C. A. P. t. group: Validating the use of Hospital Episode Statistics data and comparison of costing methodologies for economic evaluation: an end-of-life case study from the Cluster randomised triAl of PSA testing for Prostate cancer (CAP). BMJ Open 6(4), e011063 (2016)CrossRef
37.
go back to reference Turner, E.L., Metcalfe, C., Donovan, J.L., Noble, S., Sterne, J.A., Lane, J.A., Avery, K.N., Down, L., Walsh, E., Davis, M., Ben-Shlomo, Y., Oliver, S.E., Evans, S., Brindle, P., Williams, N.J., Hughes, L.J., Hill, E.M., Davies, C., Ng, S.Y., Neal, D.E., Hamdy, F.C., Martin, R.M., C. A. P. t. Group: Design and preliminary recruitment results of the Cluster randomised triAl of PSA testing for Prostate cancer (CAP). Br. J. Cancer 110(12), 2829–2836 (2014)CrossRef Turner, E.L., Metcalfe, C., Donovan, J.L., Noble, S., Sterne, J.A., Lane, J.A., Avery, K.N., Down, L., Walsh, E., Davis, M., Ben-Shlomo, Y., Oliver, S.E., Evans, S., Brindle, P., Williams, N.J., Hughes, L.J., Hill, E.M., Davies, C., Ng, S.Y., Neal, D.E., Hamdy, F.C., Martin, R.M., C. A. P. t. Group: Design and preliminary recruitment results of the Cluster randomised triAl of PSA testing for Prostate cancer (CAP). Br. J. Cancer 110(12), 2829–2836 (2014)CrossRef
38.
go back to reference Walker, E., Nowacki, A.S.: Understanding equivalence and noninferiority testing. J Gener Int Med. 26(2), 192–196 (2011) Walker, E., Nowacki, A.S.: Understanding equivalence and noninferiority testing. J Gener Int Med. 26(2), 192–196 (2011)
Metadata
Title
A methodological framework for assessing agreement between cost-effectiveness outcomes estimated using alternative sources of data on treatment costs and effects for trial-based economic evaluations
Authors
Felix Achana
Stavros Petrou
Kamran Khan
Amadou Gaye
Neena Modi
On behalf of Medicines for Neonates Investigators
Publication date
01-01-2018
Publisher
Springer Berlin Heidelberg
Published in
The European Journal of Health Economics / Issue 1/2018
Print ISSN: 1618-7598
Electronic ISSN: 1618-7601
DOI
https://doi.org/10.1007/s10198-017-0868-8

Other articles of this Issue 1/2018

The European Journal of Health Economics 1/2018 Go to the issue