Skip to main content
Key Specialties
Anesthesiology Cardiology Dermatology Diabetology Endocrinology Gastroenterology General Medicine Geriatrics Hematology Infectious Diseases Internal Medicine Nephrology Neurology Obstetrics and Gynecology Oncology Pediatrics Psychiatry Radiology Respiratory Medicine Rheumatology Surgery Urology
Congress Coverage
2024 ESMO Congress 2024 ERS Congress 2024 EASD Congress 2024 ESC Congress 2024 EAN Congress 2024 ASCO Annual Meeting 2024 ACC Congress
Clinical Collections
Advances in Alzheimer’s

EASD 2024

Keep up to date with the top stories from the 60th EASD Annual Meeting with our congress hub page.
ADVANCED SEARCH
Log in
Top
Applied Health Economics and Health Policy
Published in:

Open Access 16-05-2024 | Breast Cancer | Original Research Article

A structured process for the validation of a decision-analytic model: application to a cost-effectiveness model for risk-stratified national breast screening

Authors: Stuart J. Wright, Ewan Gray, Gabriel Rogers, Anna Donten, Katherine Payne

Published in: Applied Health Economics and Health Policy | Issue 4/2024

Login to get access

Abstract

Background

Decision-makers require knowledge of the strengths and weaknesses of decision-analytic models used to evaluate healthcare interventions to be able to confidently use the results of such models to inform policy. A number of aspects of model validity have previously been described, but no systematic approach to assessing the validity of a model has been proposed. This study aimed to consolidate the different aspects of model validity into a step-by-step approach to assessing the strengths and weaknesses of a decision-analytic model.

Methods

A pre-defined set of steps were used to conduct the validation process of an exemplar early decision-analytic-model-based cost-effectiveness analysis of a risk-stratified national breast cancer screening programme [UK healthcare perspective; lifetime horizon; costs (£; 2021)]. Internal validation was assessed in terms of descriptive validity, technical validity and face validity. External validation was assessed in terms of operational validation, convergent validity (or corroboration) and predictive validity.

Results

The results outline the findings of each step of internal and external validation of the early decision-analytic-model and present the validated model (called ‘MANC-RISK-SCREEN’). The positive aspects in terms of meeting internal validation requirements are shown together with the remaining limitations of MANC-RISK-SCREEN.

Conclusion

Following a transparent and structured validation process, MANC-RISK-SCREEN has been shown to have satisfactory internal and external validity for use in informing resource allocation decision-making. We suggest that MANC-RISK-SCREEN can be used to assess the cost-effectiveness of exemplars of risk-stratified national breast cancer screening programmes (NBSP) from the UK perspective.

Implications

A step-by-step process for conducting the validation of a decision-analytic model was developed for future use by health economists. Using this approach may help researchers to fully demonstrate the strengths and limitations of their model to decision-makers.
Appendix
Available only for authorised users
Literature
1.
Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1 on behalf of the ISPOR-SMDM Modeling Good Research Practices Task Force. Value Health. 2012;15.
2.
3.
Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, et al. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Value Health. 2012;15:812–20.CrossRefPubMed
4.
Karnon J, Stahl J, Brennan A, Caro JJ, Mar J, Möller J. Modeling using discrete event simulation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-4. Value Health J Int Soc Pharmacoecon Outcomes Res. 2012;15:821–7.CrossRef
5.
Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Mak. 2012;32:722–32.CrossRef
6.
Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Med Decis Mak. 2012;32:733–43.CrossRef
7.
Kent S, Becker F, Feenstra T, Tran-Duy A, Schlackow I, Tew M, et al. The challenge of transparency and validation in health economic decision modelling: a view from Mount Hood. Pharmacoeconomics. 2019;37:1305–12.CrossRefPubMedPubMedCentral
8.
Emerson J, Bacon R, Kent A, Neumann PJ, Cohen JT. Publication of decision model source code: attitudes of health economics authors. Pharmacoeconomics. 2019;37:1409.CrossRefPubMedPubMedCentral
9.
Sampson CJ, Arnold R, Bryan S, Clarke P, Ekins S, Hatswell A, et al. Transparency in decision modelling: what, why, who and how? Pharmacoeconomics. 2019;37:1355–69.CrossRefPubMedPubMedCentral
10.
Alarid-Escudero F, Krijkamp EM, Pechlivanoglou P, Jalal H, Kao SYZ, Yang A, et al. A need for change! A coding framework for improving transparency in decision modeling. Pharmacoeconomics. 2019;37:1329–39.CrossRefPubMedPubMedCentral
11.
Vemer P, Corro Ramos I, van Voorn GAK, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. Pharmacoeconomics. 2016;34:349–61.CrossRefPubMed
12.
Büyükkaramikli NC, Rutten-van Mölken MPMH, Severens JL, Al M. TECH-VER: a verification checklist to reduce errors in models and improve their credibility. Pharmacoeconomics. 2019;37:1391–408.CrossRefPubMedPubMedCentral
13.
McCabe C, Dixon S. Testing the validity of cost-effectiveness models. Pharmacoeconomics. 2000;17:501–13.CrossRefPubMed
14.
Nair V, Auger S, Kochanny S, Howard FM, Ginat D, Pasternak-Wise O, et al. Development and validation of a decision analytical model for posttreatment surveillance for patients with oropharyngeal carcinoma. JAMA Netw Open. 2022;5:e227240–e227240.CrossRefPubMedPubMedCentral
15.
16.
Godley KC, Gladwell C, Murray PJ, Denton E. The UK breast screening program – what you need to know. Climacteric. 2017;20:313–20.CrossRefPubMed
17.
Tyrer J, Duuy SW, Cuzick J. A breast cancer prediction model incorporating familial and personal risk factors. Stat Med. 2004;23:1111–30.CrossRefPubMed
18.
Lee A, Mavaddat N, Wilcox AN, Cunningham AP, Carver T, Hartley S, et al. BOADICEA: a comprehensive breast cancer risk prediction model incorporating genetic and nongenetic risk factors. Genet Med. 2019;21:1708–18.CrossRefPubMedPubMedCentral
19.
Evans DGR, Howell A. Breast cancer risk-assessment models. Breast Cancer Res. 2007;9:1–8.CrossRef
20.
Pashayan N, Antoniou AC, Ivanus U, Esserman LJ, Easton DF, French D, et al. Personalized early detection and prevention of breast cancer: ENVISION consensus statement. Nat Rev Clin Oncol. 2020;17:687–705.CrossRefPubMedPubMedCentral
21.
Evans DGR, Warwick J, Astley SM, Stavrinos P, Sahin S, Ingham S, et al. Assessing individual breast cancer risk within the U.K. National Health Service Breast Screening Program: a new paradigm for cancer prevention. Cancer Prev Res (Phila Pa). 2012;5:943–51.CrossRef
22.
Esserman LJ. The WISDOM Study: breaking the deadlock in the breast cancer screening debate. NPJ Breast Cancer. 2017;3:1–7.CrossRef
23.
Roux A, Cholerton R, Sicsic J, Moumjid N, French DP, Giorgi Rossi P, et al. Study protocol comparing the ethical, psychological and socio-economic impact of personalised breast cancer screening to that of standard screening in the “My Personal Breast Screening” (MyPeBS) randomised clinical trial. BMC Cancer. 2022. p. 22.
24.
Evans DG, Astley S, Stavrinos P, Harkness E, Donnelly LS, Dawe S, et al. Improvement in risk prediction, early detection and prevention of breast cancer in the NHS Breast Screening Programme and family history clinics: a dual cohort study. Programme Grants Appl Res. 2016;4:1–210.CrossRef
25.
French DP, Astley S, Astley S, Brentnall AR, Cuzick J, Dobrashian R, et al. What are the benefits and harms of risk stratified screening as part of the NHS breast screening programme? Study protocol for a multi-site non-randomised comparison of BC-predict versus usual screening (NCT04359420). BMC Cancer. 2020;20:1–14.CrossRef
26.
Amir E, Evans DG, Shenton A, Lalloo F, Moran A, Boggis C, et al. Evaluation of breast cancer risk assessment packages in the family history evaluation and screening programme. J Med Genet. 2003;40:807–14.CrossRefPubMedPubMedCentral
27.
28.
Brentnall AR, Harkness EF, Astley SM, Donnelly LS, Stavrinos P, Sampson S, et al. Mammographic density adds accuracy to both the Tyrer-Cuzick and Gail breast cancer risk models in a prospective UK screening cohort. Breast Cancer Res BCR [Internet]. 2015 [cited 2022 Feb 24];17. https://​pubmed.​ncbi.​nlm.​nih.​gov/​26627479/​
29.
Clift AK, Dodwell D, Lord S, Petrou S, Brady SM, Collins GS, et al. The current status of risk-stratified breast screening. Br J Cancer. 2021;126:533–50.CrossRefPubMedPubMedCentral
30.
Sculpher MJ, Claxton K, Drummond M, McCabe C. Whither trial-based economic evaluation for health care decision making? Health Econ. 2006;15:677–87.CrossRefPubMed
31.
Akehurst R, Anderson P, Brazier J, Brennan A, Briggs A, Buxton M, et al. Decision analytic modelling in the economic evaluation of health technologies. Pharmacoeconomics. 2000;17:443–4.CrossRef
32.
McGuire A, Morris S. What is it to be a model? Trials and tribulations in economic evaluation. Health Econ Prev Care. 2000;1:33–6.CrossRef
33.
Buxton MJ, Drummond MF, Van Hout BA, Prince RL, Sheldon TA, Szucs T, et al. Modelling in economic evaluation: an unavoidable fact of life. Health Econ. 1997;6:217–27.CrossRefPubMed
34.
Tappenden P, Chilcott JB. Avoiding and identifying errors and other threats to the credibility of health economic models. Pharmacoeconomics. 2014;32:967–79.CrossRefPubMed
35.
Sculpher M, Drummond M, Buxton M. The iterative use of economic evaluation as part of the process of health technology assessment. J Health Serv Res Policy. 1997;2:26–30.CrossRefPubMed
36.
Gray E, Donten A, Karssemeijer N, van Gils C, Evans DG, Astley S, et al. Evaluation of a stratified national breast screening program in the United Kingdom: an early model-based cost-effectiveness analysis. Value Health. 2017;20:1100–9.CrossRefPubMed
37.
Hammerschmidt T, Goertz A, Wagenpfeil S, Neiss A, Wutzler P, Banz K. Validation of health economic models: the example of EVITA. Value Health. 2003;6:551–9.CrossRefPubMed
38.
Haji Ali Afzali H, Gray J, Karnon J. Model performance evaluation (validation and calibration) in model-based studies of therapeutic interventions for cardiovascular diseases. Appl Health Econ Health Policy. 2013;11:85–93.
39.
Mcwilliams L, Gareth Evans D, Payne K, Harrison F, Howell A, Howell SJ, et al. Implementing risk-stratified breast screening in England: an agenda setting meeting. Cancers. 2022;14:4636.CrossRefPubMedPubMedCentral
40.
Jones J, Hunter D. Consensus methods for medical and health services research. Br Med J. 1995;311:376–80.CrossRef
41.
42.
43.
Wright SJ, Eden M, Ruane H, Byers H, Evans DG, Harvie M, et al. Estimating the cost of 3 risk prediction strategies for potential use in the United Kingdom National Breast Screening Program. Med Decis Mak Policy Pract. 2023;8:238146832311713.
44.
45.
NHS Digital Screening and Immunisations Team. Breast Screening Programme. NHS Digital; 2021.
46.
47.
Evans DG, van Veen EM, Byers H, Roberts E, Howell A, Howell SJ, et al. The importance of ethnicity: are breast cancer polygenic risk scores ready for women who are not of white European origin? Int J Cancer. 2022;150:73–9.CrossRefPubMed
48.
49.
Kollias J, Murphy CA, Elston CW, Ellis IO, Robertson JFR, Blarney RW. The prognosis of small primary breast cancers. Eur J Cancer. 1999;35:908–12.CrossRefPubMed
50.
Wen J, Ye F, Li S, Huang X, Yang L, Xiao X, et al. The practicability of a novel prognostic index (PI) model and comparison with Nottingham Prognostic Index (NPI) in stage I-III breast cancer patients undergoing surgical treatment. PLoS ONE. 2015;10: e0143537.CrossRefPubMedPubMedCentral
51.
Cheng L, Al-Kaisi NK, Gordon NH, Liu AY, Gebrail F, Shenk RR. Relationship between the size and margin status of ductal carcinoma in situ of the breast and residual disease. JNCI J Natl Cancer Inst. 1997;89:1356–60.CrossRefPubMed
52.
53.
Pashayan N, Morris S, Gilbert FJ, Pharoah PDP. Cost-effectiveness and benefit-to-harm ratio of risk-stratified screening for breast cancer: a life-table model. JAMA Oncol. 2018;4:1504–10.CrossRefPubMedPubMedCentral
Metadata
Title
A structured process for the validation of a decision-analytic model: application to a cost-effectiveness model for risk-stratified national breast screening
Authors
Stuart J. Wright
Ewan Gray
Gabriel Rogers
Anna Donten
Katherine Payne
Publication date
16-05-2024
Publisher
Springer International Publishing
Published in
Applied Health Economics and Health Policy / Issue 4/2024
Print ISSN: 1175-5652
Electronic ISSN: 1179-1896
DOI
https://doi.org/10.1007/s40258-024-00887-z

Other articles of this Issue 4/2024

Applied Health Economics and Health Policy 4/2024 Go to the issue

Systematic Review

Economic Evaluation of COVID-19 Immunization Strategies: A Systematic Review and Narrative Synthesis

Systematic Review

Cost of Low-Value Imaging Worldwide: A Systematic Review

Original Research Article

Cost Effectiveness of Adding Fenfluramine to Standard of Care for Patients with Dravet Syndrome in Sweden

Systematic Review

Measurement of Catastrophic Health Expenditure in India: A Systematic Review and Meta-Analysis

Systematic Review

Model-Based Economic Evaluations of Interventions for Dementia: An Updated Systematic Review and Quality Assessment

Original Research Article

Cost-Effectiveness of Novel Agent Regimens for Transplant-Eligible Newly Diagnosed Multiple Myeloma Patients in India