Skip to main content
Top
Published in: PharmacoEconomics 12/2019

01-12-2019 | Stroke | Original Research Article

Performance of the UKPDS Outcomes Model 2 for Predicting Death and Cardiovascular Events in Patients with Type 2 Diabetes Mellitus from a German Population-Based Cohort

Authors: Michael Laxy, Verena Maria Schöning, Christoph Kurz, Rolf Holle, Annette Peters, Christa Meisinger, Wolfgang Rathmann, Kristin Mühlenbruch, Katharina Kähm

Published in: PharmacoEconomics | Issue 12/2019

Login to get access

Abstract

Background and Objective

Accurate prediction of relevant outcomes is important for targeting therapies and to support health economic evaluations of healthcare interventions in patients with diabetes. The United Kingdom Prospective Diabetes Study (UKPDS) risk equations are some of the most frequently used risk equations. This study aims to analyze the calibration and discrimination of the updated UKPDS risk equations as implemented in the UKPDS Outcomes Model 2 (UKPDS-OM2) for predicting cardiovascular (CV) events and death in patients with type 2 diabetes mellitus (T2DM) from population-based German samples.

Methods

Analyses are based on data of 456 individuals diagnosed with T2DM who participated in two population-based studies in southern Germany (KORA (Cooperative Health Research in the Region of Augsburg)-A: 1997/1998, n = 178; KORA-S4: 1999–2001, n = 278). We compared the participants’ 10-year observed incidence of mortality, CV mortality, myocardial infarction (MI), and stroke with the predicted event rate of the UKPDS-OM2. The model’s calibration was evaluated by Greenwood–Nam–D’Agostino tests and discrimination was evaluated by C-statistics.

Results

Of the 456 participants with T2DM (mean age 65 years, mean diabetes duration 8 years, 56% male), over the 10-year follow-up time 129 died (61 due to CV events), 64 experienced an MI, and 46 a stroke. The UKPDS-OM2 significantly over-predicted mortality and CV mortality by 25% and 28%, respectively (Greenwood–Nam–D’Agostino tests: p < 0.01), but there was no significant difference between predicted and observed MI and stroke risk. The model poorly discriminated for death (C-statistic [95% confidence interval] = 0.64 [0.60–0.69]), CV death (0.64 [0.58–0.71]), and MI (0.58 [0.52–0.66]), and failed to discriminate for stroke (0.57 [0.47–0.66]).

Conclusions

The study results demonstrate acceptable calibration and poor discrimination of the UKPDS-OM2 for predicting death and CV events in this population-based German sample. Those limitations should be considered when using the UKPDS-OM2 for economic evaluations of healthcare strategies or using the risk equations for clinical decision-making.
Appendix
Available only for authorised users
Literature
1.
go back to reference Jacobs E, Hoyer A, Brinks R, Kuss O, Rathmann W. Burden of mortality attributable to diagnosed diabetes: a nationwide analysis based on claims data from 65 million people in Germany. Diabetes Care. 2017;40(12):1703–9.CrossRef Jacobs E, Hoyer A, Brinks R, Kuss O, Rathmann W. Burden of mortality attributable to diagnosed diabetes: a nationwide analysis based on claims data from 65 million people in Germany. Diabetes Care. 2017;40(12):1703–9.CrossRef
2.
go back to reference Bommer C, Sagalova V, Heesemann E, Manne-Goehler J, Atun R, Barnighausen T, et al. Global economic burden of diabetes in adults: projections from 2015 to 2030. Diabetes Care. 2018;41(5):963–70.CrossRef Bommer C, Sagalova V, Heesemann E, Manne-Goehler J, Atun R, Barnighausen T, et al. Global economic burden of diabetes in adults: projections from 2015 to 2030. Diabetes Care. 2018;41(5):963–70.CrossRef
3.
go back to reference Hunger M, Thorand B, Schunk M, Doring A, Menn P, Peters A, et al. Multimorbidity and health-related quality of life in the older population: results from the German KORA-age study. Health Qual Life Outcomes. 2011;9:53.CrossRef Hunger M, Thorand B, Schunk M, Doring A, Menn P, Peters A, et al. Multimorbidity and health-related quality of life in the older population: results from the German KORA-age study. Health Qual Life Outcomes. 2011;9:53.CrossRef
4.
go back to reference American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27(9):2262–5.CrossRef American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27(9):2262–5.CrossRef
5.
go back to reference Baena-Diez JM, Peñafiel J, Subirana I, Ramos R, Elosua R, Marín-Ibañez A, FRESCO Investigators, et al. Risk of cause-specific death in individuals with diabetes: a competing risks analysis. Diabetes Care. 2016;39(11):1987–95.CrossRef Baena-Diez JM, Peñafiel J, Subirana I, Ramos R, Elosua R, Marín-Ibañez A, FRESCO Investigators, et al. Risk of cause-specific death in individuals with diabetes: a competing risks analysis. Diabetes Care. 2016;39(11):1987–95.CrossRef
7.
go back to reference Chou R, Dana T, Blazina I, Daeges M, Bougatsos C, Grusing S, et al. Statin use for the prevention of cardiovascular disease in adults: a systematic review for the U.S. Preventive Services Task Force. Rockville: Agency for Healthcare Research and Quality (US); 2016. (Evidence Syntheses, No. 139). https://www.ncbi.nlm.nih.gov/books/NBK396415/. Accessed 3 Sept 2018. Chou R, Dana T, Blazina I, Daeges M, Bougatsos C, Grusing S, et al. Statin use for the prevention of cardiovascular disease in adults: a systematic review for the U.S. Preventive Services Task Force. Rockville: Agency for Healthcare Research and Quality (US); 2016. (Evidence Syntheses, No. 139). https://​www.​ncbi.​nlm.​nih.​gov/​books/​NBK396415/​. Accessed 3 Sept 2018.
8.
go back to reference Govan L, Wu O, Lindsay R, Briggs A. How do diabetes models measure up? A review of diabetes economic models and ADA guidelines. J Health Econ Outcomes Res. 2015;3(2):132–52. Govan L, Wu O, Lindsay R, Briggs A. How do diabetes models measure up? A review of diabetes economic models and ADA guidelines. J Health Econ Outcomes Res. 2015;3(2):132–52.
9.
go back to reference UK Prospective Diabetes Study (UKPDS). VIII. Study design, progress and performance. Diabetologia. 1991;34(12):877–90.CrossRef UK Prospective Diabetes Study (UKPDS). VIII. Study design, progress and performance. Diabetologia. 1991;34(12):877–90.CrossRef
10.
go back to reference Hayes AJ, Leal J, Gray AM, Holman RR, Clarke PM. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56(9):1925–33.CrossRef Hayes AJ, Leal J, Gray AM, Holman RR, Clarke PM. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56(9):1925–33.CrossRef
11.
go back to reference McEwan P, Bergenheim K, Currie CJ. External validation of the UKPDS Oucomes model equations (UKPDS 68), and the UKPDS risk engine equations (UKPDS 56 and 60) in forecasting cardiovascular outcomes in people with type 2 diabetes. Value Health. 2006;9(3):A37–8.CrossRef McEwan P, Bergenheim K, Currie CJ. External validation of the UKPDS Oucomes model equations (UKPDS 68), and the UKPDS risk engine equations (UKPDS 56 and 60) in forecasting cardiovascular outcomes in people with type 2 diabetes. Value Health. 2006;9(3):A37–8.CrossRef
12.
go back to reference Simmons RK, Coleman RL, Price HC, Holman RR, Khaw K-T, Wareham NJ, et al. Performance of the UK Prospective Diabetes Study Risk Engine and the Framingham risk equations in estimating cardiovascular disease in the EPIC-Norfolk cohort. Diabetes Care. 2009;32(4):708–13.CrossRef Simmons RK, Coleman RL, Price HC, Holman RR, Khaw K-T, Wareham NJ, et al. Performance of the UK Prospective Diabetes Study Risk Engine and the Framingham risk equations in estimating cardiovascular disease in the EPIC-Norfolk cohort. Diabetes Care. 2009;32(4):708–13.CrossRef
13.
go back to reference Tao L, Wilson ECF, Griffin SJ, Simmons RK. Performance of the UKPDS outcomes model for prediction of myocardial infarction and stroke in the ADDITION-Europe trial cohort. Value Health. 2013;16(6):1074–80.CrossRef Tao L, Wilson ECF, Griffin SJ, Simmons RK. Performance of the UKPDS outcomes model for prediction of myocardial infarction and stroke in the ADDITION-Europe trial cohort. Value Health. 2013;16(6):1074–80.CrossRef
14.
go back to reference van Dieren S, Peelen LM, Nöthlings U, van der Schouw YT, Rutten GE, Spijkerman AM, et al. External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes. Diabetologia. 2011;54(2):264–70.CrossRef van Dieren S, Peelen LM, Nöthlings U, van der Schouw YT, Rutten GE, Spijkerman AM, et al. External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes. Diabetologia. 2011;54(2):264–70.CrossRef
15.
go back to reference Kengne AP, Patel A, Colagiuri S, Heller S, Hamet P, Marre M, ADVANCE Collaborative Group, et al. The Framingham and UK Prospective Diabetes Study (UKPDS) risk equations do not reliably estimate the probability of cardiovascular events in a large ethnically diverse sample of patients with diabetes: the Action in Diabetes and Vascular Disease: Preterax and Diamicron-MR Controlled Evaluation (ADVANCE) study. Diabetologia. 2010;53(5):821–31.CrossRef Kengne AP, Patel A, Colagiuri S, Heller S, Hamet P, Marre M, ADVANCE Collaborative Group, et al. The Framingham and UK Prospective Diabetes Study (UKPDS) risk equations do not reliably estimate the probability of cardiovascular events in a large ethnically diverse sample of patients with diabetes: the Action in Diabetes and Vascular Disease: Preterax and Diamicron-MR Controlled Evaluation (ADVANCE) study. Diabetologia. 2010;53(5):821–31.CrossRef
17.
go back to reference Willis M, Johansen P, Nilsson A, Asseburg C. Validation of the economic and health outcomes model of type 2 diabetes mellitus (ECHO-T2DM). Pharmacoeconomics. 2017;35(3):375–96.CrossRef Willis M, Johansen P, Nilsson A, Asseburg C. Validation of the economic and health outcomes model of type 2 diabetes mellitus (ECHO-T2DM). Pharmacoeconomics. 2017;35(3):375–96.CrossRef
18.
go back to reference Lundqvist A, Steen Carlsson K, Johansen P, Andersson E, Willis M. Validation of the IHE Cohort Model of Type 2 Diabetes and the impact of choice of macrovascular risk equations. PLoS One. 2014;9(10):e110235.CrossRef Lundqvist A, Steen Carlsson K, Johansen P, Andersson E, Willis M. Validation of the IHE Cohort Model of Type 2 Diabetes and the impact of choice of macrovascular risk equations. PLoS One. 2014;9(10):e110235.CrossRef
19.
go back to reference McEwan P, Foos V, Palmer JL, Lamotte M, Lloyd A, Grant D. Validation of the IMS CORE Diabetes Model. Value Health. 2014;17(6):714–24.CrossRef McEwan P, Foos V, Palmer JL, Lamotte M, Lloyd A, Grant D. Validation of the IMS CORE Diabetes Model. Value Health. 2014;17(6):714–24.CrossRef
20.
go back to reference McEwan P, Ward T, Bennett H, Bergenheim K. Validation of the UKPDS 82 risk equations within the Cardiff Diabetes Model. Cost Effect Resour Alloc. 2015;13:12.CrossRef McEwan P, Ward T, Bennett H, Bergenheim K. Validation of the UKPDS 82 risk equations within the Cardiff Diabetes Model. Cost Effect Resour Alloc. 2015;13:12.CrossRef
21.
go back to reference Palmer AJ, Mount Hood 5 Modeling Group, Clarke P, Gray A, Leal J, Lloyd A, et al. Computer modeling of diabetes and its complications: a report on the Fifth Mount Hood challenge meeting. Value Health. 2013;16(4):670–85.CrossRef Palmer AJ, Mount Hood 5 Modeling Group, Clarke P, Gray A, Leal J, Lloyd A, et al. Computer modeling of diabetes and its complications: a report on the Fifth Mount Hood challenge meeting. Value Health. 2013;16(4):670–85.CrossRef
22.
go back to reference Basu S, Sussman JB, Berkowitz SA, Hayward RA, Bertoni AG, Correa A, et al. Validation of Risk Equations for Complications of Type 2 Diabetes (RECODe) using individual participant data from diverse longitudinal cohorts in the U.S. Diabetes Care. 2018;41(3):586–95.CrossRef Basu S, Sussman JB, Berkowitz SA, Hayward RA, Bertoni AG, Correa A, et al. Validation of Risk Equations for Complications of Type 2 Diabetes (RECODe) using individual participant data from diverse longitudinal cohorts in the U.S. Diabetes Care. 2018;41(3):586–95.CrossRef
23.
go back to reference Holle R, Happich M, Lowel H, Wichmann HE, MONICA/KORA Study Group. KORA—a research platform for population based health research. Gesundheitswesen. 2005;67(Suppl 1):S19–25.CrossRef Holle R, Happich M, Lowel H, Wichmann HE, MONICA/KORA Study Group. KORA—a research platform for population based health research. Gesundheitswesen. 2005;67(Suppl 1):S19–25.CrossRef
24.
go back to reference Laxy M, Knoll G, Schunk M, Meisinger C, Huth C, Holle R. Quality of diabetes care in Germany improved from 2000 to 2007 to 2014, but improvements diminished since 2007. Evidence from the population-based KORA studies. PLoS One. 2016;11(10):e0164704.CrossRef Laxy M, Knoll G, Schunk M, Meisinger C, Huth C, Holle R. Quality of diabetes care in Germany improved from 2000 to 2007 to 2014, but improvements diminished since 2007. Evidence from the population-based KORA studies. PLoS One. 2016;11(10):e0164704.CrossRef
25.
go back to reference Laxy M, Mielck A, Hunger M, Schunk M, Meisinger C, Ruckert IM, et al. The association between patient-reported self-management behavior, intermediate clinical outcomes, and mortality in patients with type 2 diabetes: results from the KORA-A study. Diabetes Care. 2014;37(6):1604–12.CrossRef Laxy M, Mielck A, Hunger M, Schunk M, Meisinger C, Ruckert IM, et al. The association between patient-reported self-management behavior, intermediate clinical outcomes, and mortality in patients with type 2 diabetes: results from the KORA-A study. Diabetes Care. 2014;37(6):1604–12.CrossRef
26.
go back to reference Yuan Y. Multiple imputation using SAS software. J Stat Softw. 2011;45(6):1–25.CrossRef Yuan Y. Multiple imputation using SAS software. J Stat Softw. 2011;45(6):1–25.CrossRef
28.
go back to reference Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med. 2015;34(10):1659–80.CrossRef Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med. 2015;34(10):1659–80.CrossRef
29.
go back to reference Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: validating a prognostic model. BMJ (Clin Res Ed). 2009;338:b605.CrossRef Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: validating a prognostic model. BMJ (Clin Res Ed). 2009;338:b605.CrossRef
30.
go back to reference Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38.CrossRef Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38.CrossRef
31.
go back to reference Pencina MJ, D’Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23(13):2109–23.CrossRef Pencina MJ, D’Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23(13):2109–23.CrossRef
32.
go back to reference Olchanski N, Cohen JT, Neumann PJ, Wong JB, Kent DM. Understanding the value of individualized information: the impact of poor calibration or discrimination in outcome prediction models. Med Decis Making. 2017;37(7):790–801.CrossRef Olchanski N, Cohen JT, Neumann PJ, Wong JB, Kent DM. Understanding the value of individualized information: the impact of poor calibration or discrimination in outcome prediction models. Med Decis Making. 2017;37(7):790–801.CrossRef
33.
go back to reference Kähm K, Laxy M, Schneider U, Rogowski WH, Lhachimi SK, Holle R. Health care costs associated with incident complications in patients with type 2 diabetes in Germany. Diabetes Care. 2018;41(5):971–8.CrossRef Kähm K, Laxy M, Schneider U, Rogowski WH, Lhachimi SK, Holle R. Health care costs associated with incident complications in patients with type 2 diabetes in Germany. Diabetes Care. 2018;41(5):971–8.CrossRef
Metadata
Title
Performance of the UKPDS Outcomes Model 2 for Predicting Death and Cardiovascular Events in Patients with Type 2 Diabetes Mellitus from a German Population-Based Cohort
Authors
Michael Laxy
Verena Maria Schöning
Christoph Kurz
Rolf Holle
Annette Peters
Christa Meisinger
Wolfgang Rathmann
Kristin Mühlenbruch
Katharina Kähm
Publication date
01-12-2019
Publisher
Springer International Publishing
Published in
PharmacoEconomics / Issue 12/2019
Print ISSN: 1170-7690
Electronic ISSN: 1179-2027
DOI
https://doi.org/10.1007/s40273-019-00822-4

Other articles of this Issue 12/2019

PharmacoEconomics 12/2019 Go to the issue