Skip to main content
Top
Published in: PharmacoEconomics 4/2016

Open Access 01-04-2016 | Original Research Article

AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users

Authors: P. Vemer, I. Corro Ramos, G. A. K. van Voorn, M. J. Al, T. L. Feenstra

Published in: PharmacoEconomics | Issue 4/2016

Login to get access

Abstract

Background

A trade-off exists between building confidence in health-economic (HE) decision models and the use of scarce resources. We aimed to create a practical tool providing model users with a structured view into the validation status of HE decision models, to address this trade-off.

Methods

A Delphi panel was organized, and was completed by a workshop during an international conference. The proposed tool was constructed iteratively based on comments from, and the discussion amongst, panellists. During the Delphi process, comments were solicited on the importance and feasibility of possible validation techniques for modellers, their relevance for decision makers, and the overall structure and formulation in the tool.

Results

The panel consisted of 47 experts in HE modelling and HE decision making from various professional and international backgrounds. In addition, 50 discussants actively engaged in the discussion at the conference workshop and returned 19 questionnaires with additional comments. The final version consists of 13 items covering all relevant aspects of HE decision models: the conceptual model, the input data, the implemented software program, and the model outcomes.

Conclusions

Assessment of the Validation Status of Health-Economic decision models (AdViSHE) is a validation-assessment tool in which model developers report in a systematic way both on validation efforts performed and on their outcomes. Subsequently, model users can establish whether confidence in the model is justified or whether additional validation efforts should be undertaken. In this way, AdViSHE enhances transparency of the validation status of HE models and supports efficient model validation.
Appendix
Available only for authorised users
Literature
1.
go back to reference Caro JJ, Briggs AH, Siebert U, Kuntz KM. ISPOR-SMDM Modeling Good Research Practices Task force. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Value Health. 2012;15(6):796–803.CrossRefPubMed Caro JJ, Briggs AH, Siebert U, Kuntz KM. ISPOR-SMDM Modeling Good Research Practices Task force. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Value Health. 2012;15(6):796–803.CrossRefPubMed
2.
go back to reference Balci O. Golden rules of verification, validation, testing, and certification of modeling and simulation applications. SCS M&S Mag 2010;4. Balci O. Golden rules of verification, validation, testing, and certification of modeling and simulation applications. SCS M&S Mag 2010;4.
3.
go back to reference Caro JJ, Eddy DM, Kan H, Kaltz C, Patel B, Eldessouki R, Briggs AH; ISPOR-AMCP-NPC Modeling CER Task Forces. Questionnaire to assess relevance and credibility of modeling studies for informing health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force Report. Value Health. 2014;17(2):174–82.CrossRef Caro JJ, Eddy DM, Kan H, Kaltz C, Patel B, Eldessouki R, Briggs AH; ISPOR-AMCP-NPC Modeling CER Task Forces. Questionnaire to assess relevance and credibility of modeling studies for informing health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force Report. Value Health. 2014;17(2):174–82.CrossRef
4.
go back to reference Sargent RG. Validation and verification of simulation models. In: Ingalls R, Rossetti M, Smith J, Peters B, editors. 2004 Winter Simulation Conference Piscataway, NJ. IEEE Press; 2004. Sargent RG. Validation and verification of simulation models. In: Ingalls R, Rossetti M, Smith J, Peters B, editors. 2004 Winter Simulation Conference Piscataway, NJ. IEEE Press; 2004.
5.
go back to reference Law AM. How to build valid and credible simulation models. Peters B, Smith J, Medeiros D, Rohrer M, editors. 2001 Winter Simulation Conference Arlington, VA. IEEE Press; 2001. Law AM. How to build valid and credible simulation models. Peters B, Smith J, Medeiros D, Rohrer M, editors. 2001 Winter Simulation Conference Arlington, VA. IEEE Press; 2001.
6.
go back to reference Law AM, Kelton WD. Simulation modeling and analysis. 4th ed. New York: McGraw-Hill; 2007. Law AM, Kelton WD. Simulation modeling and analysis. 4th ed. New York: McGraw-Hill; 2007.
8.
go back to reference Evers S, Goossens M, de Vet H, van Tulder M, Ament A. Criteria list for assessment of methodological quality of economic evaluations: consensus on health economic criteria. Int J Technol Assess Health Care. 2005;21(2):240–5.PubMed Evers S, Goossens M, de Vet H, van Tulder M, Ament A. Criteria list for assessment of methodological quality of economic evaluations: consensus on health economic criteria. Int J Technol Assess Health Care. 2005;21(2):240–5.PubMed
9.
go back to reference Philips Z, Bojke L, Sculpher M, Claxton K, Golder S. Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. Pharmacoeconomics. 2006;24(4):355–71.CrossRefPubMed Philips Z, Bojke L, Sculpher M, Claxton K, Golder S. Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. Pharmacoeconomics. 2006;24(4):355–71.CrossRefPubMed
10.
go back to reference Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value Health. 2012;15(6):843–50.CrossRefPubMed Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value Health. 2012;15(6):843–50.CrossRefPubMed
11.
12.
go back to reference Vemer P, van Voorn GA, Ramos IC, Krabbe PF, Al MJ, Feenstra TL. Improving model validation in health technology assessment: comments on guidelines of the ISPOR-SMDM Modeling Good Research Practices Task Force. Value Health. 2013;16(6):1106–7.CrossRefPubMed Vemer P, van Voorn GA, Ramos IC, Krabbe PF, Al MJ, Feenstra TL. Improving model validation in health technology assessment: comments on guidelines of the ISPOR-SMDM Modeling Good Research Practices Task Force. Value Health. 2013;16(6):1106–7.CrossRefPubMed
14.
go back to reference Dalkey N, Helmer-Hirschberg O. An experimental application of the Delphi method to the use of experts. Manage Sci. 1963;9(3):458–67.CrossRef Dalkey N, Helmer-Hirschberg O. An experimental application of the Delphi method to the use of experts. Manage Sci. 1963;9(3):458–67.CrossRef
15.
go back to reference Linstone HA, Turoff M, editors. The Delphi method: techniques and applications. New Jersey Institute of Technology: Newark; 2002. Linstone HA, Turoff M, editors. The Delphi method: techniques and applications. New Jersey Institute of Technology: Newark; 2002.
16.
go back to reference Whiting P, Al MJ, Westwood ME, Corro Ramos I, Ryder S, Armstrong N, Misso K, Ross J, Severens J, Kleijnen J. Viscoelastic point-of-care testing to assist with the diagnosis, management and monitoring of haemostasis: a systematic review and cost-effectiveness analysis. 2014. Whiting P, Al MJ, Westwood ME, Corro Ramos I, Ryder S, Armstrong N, Misso K, Ross J, Severens J, Kleijnen J. Viscoelastic point-of-care testing to assist with the diagnosis, management and monitoring of haemostasis: a systematic review and cost-effectiveness analysis. 2014.
17.
go back to reference Hoogendoorn M, Rutten-van Mölken MPMH, Hoogenveen RT, van Genugten MLL, Buist AS, Wouters EFM, et al. A dynamic population model of disease progression in COPD. Eur Respir J. 2005;26(2):223–33.CrossRefPubMed Hoogendoorn M, Rutten-van Mölken MPMH, Hoogenveen RT, van Genugten MLL, Buist AS, Wouters EFM, et al. A dynamic population model of disease progression in COPD. Eur Respir J. 2005;26(2):223–33.CrossRefPubMed
18.
go back to reference Halpern MT, Luce BR, Brown RE, Geneste B. Health and economic outcomes modelling practices: a suggested framework. Value Health. 1998;1(2):131–47.CrossRefPubMed Halpern MT, Luce BR, Brown RE, Geneste B. Health and economic outcomes modelling practices: a suggested framework. Value Health. 1998;1(2):131–47.CrossRefPubMed
20.
go back to reference Zeigler BP, Praehofer H, Kim TG. Theory of modeling and simulation. 2nd ed. San Diego: Academic Press; 2000. Zeigler BP, Praehofer H, Kim TG. Theory of modeling and simulation. 2nd ed. San Diego: Academic Press; 2000.
22.
go back to reference Schruben LW. Establishing the credibility of simulations. Simulation. 1980;34(3):101–5.CrossRef Schruben LW. Establishing the credibility of simulations. Simulation. 1980;34(3):101–5.CrossRef
25.
go back to reference Chilcott J, Tappenden P, Rawdin A, Johnson M, Kaltenthaler E, Paisley S, et al. Avoiding and identifying errors in health technology assessment models: qualitative study and methodological review. Health Technol Assess. 2010;14(25):iii–iv ix–xii, 1–107.CrossRefPubMed Chilcott J, Tappenden P, Rawdin A, Johnson M, Kaltenthaler E, Paisley S, et al. Avoiding and identifying errors in health technology assessment models: qualitative study and methodological review. Health Technol Assess. 2010;14(25):iii–iv ix–xii, 1–107.CrossRefPubMed
Metadata
Title
AdViSHE: A Validation-Assessment Tool of Health-Economic Models for Decision Makers and Model Users
Authors
P. Vemer
I. Corro Ramos
G. A. K. van Voorn
M. J. Al
T. L. Feenstra
Publication date
01-04-2016
Publisher
Springer International Publishing
Published in
PharmacoEconomics / Issue 4/2016
Print ISSN: 1170-7690
Electronic ISSN: 1179-2027
DOI
https://doi.org/10.1007/s40273-015-0327-2

Other articles of this Issue 4/2016

PharmacoEconomics 4/2016 Go to the issue