Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Metformin | Research article

Dynamic risk prediction for diabetes using biomarker change measurements

Authors: Layla Parast, Megan Mathews, Mark W. Friedberg

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

Dynamic risk models, which incorporate disease-free survival and repeated measurements over time, might yield more accurate predictions of future health status compared to static models. The objective of this study was to develop and apply a dynamic prediction model to estimate the risk of developing type 2 diabetes mellitus.

Methods

Both a static prediction model and a dynamic landmark model were used to provide predictions of a 2-year horizon time for diabetes-free survival, updated at 1, 2, and 3 years post-baseline i.e., predicting diabetes-free survival to 2 years and predicting diabetes-free survival to 3 years, 4 years, and 5 years post-baseline, given the patient already survived past 1 year, 2 years, and 3 years post-baseline, respectively. Prediction accuracy was evaluated at each time point using robust non-parametric procedures. Data from 2057 participants of the Diabetes Prevention Program (DPP) study (1027 in metformin arm, 1030 in placebo arm) were analyzed.

Results

The dynamic landmark model demonstrated good prediction accuracy with area under curve (AUC) estimates ranging from 0.645 to 0.752 and Brier Score estimates ranging from 0.088 to 0.135. Relative to a static risk model, the dynamic landmark model did not significantly differ in terms of AUC but had significantly lower (i.e., better) Brier Score estimates for predictions at 1, 2, and 3 years (e.g. 0.167 versus 0.099; difference − 0.068 95% CI − 0.083 to − 0.053, at 3 years in placebo group) post-baseline.

Conclusions

Dynamic prediction models based on longitudinal, repeated risk factor measurements have the potential to improve the accuracy of future health status predictions.
Appendix
Available only for authorised users
Literature
1.
go back to reference Miotto R, Li L, Kidd BA, Dudley JT. Deep patient: an unsupervised representation to predict the future of patients from the electronic health records. Sci Rep. 2016;6:26094.CrossRef Miotto R, Li L, Kidd BA, Dudley JT. Deep patient: an unsupervised representation to predict the future of patients from the electronic health records. Sci Rep. 2016;6:26094.CrossRef
2.
go back to reference Abbasi A, Peelen LM, Corpeleijn E, van der Schouw YT, Stolk RP, Spijkerman AMW, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ. 2012;345:e5900.CrossRef Abbasi A, Peelen LM, Corpeleijn E, van der Schouw YT, Stolk RP, Spijkerman AMW, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ. 2012;345:e5900.CrossRef
3.
go back to reference Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793–5.CrossRef Collins FS, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372(9):793–5.CrossRef
4.
go back to reference Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA. 2013;309(13):1351–2.CrossRef Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA. 2013;309(13):1351–2.CrossRef
5.
go back to reference Damen JAAG, Hooft L, Schuit E, Debray TPA, Collins GS, Tzoulaki I, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ. 2016;353:i2416.CrossRef Damen JAAG, Hooft L, Schuit E, Debray TPA, Collins GS, Tzoulaki I, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ. 2016;353:i2416.CrossRef
6.
go back to reference D’Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, et al. General cardiovascular risk profile for use in primary care. Circulation. 2008;117(6):743–53.CrossRef D’Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, et al. General cardiovascular risk profile for use in primary care. Circulation. 2008;117(6):743–53.CrossRef
7.
go back to reference Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, et al. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst. 1989;81(24):1879–86.CrossRef Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, et al. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst. 1989;81(24):1879–86.CrossRef
8.
go back to reference Ginsburg GS, Willard HF. Genomic and personalized medicine: foundations and applications. Transl Res. 2009;154(6):277–87.CrossRef Ginsburg GS, Willard HF. Genomic and personalized medicine: foundations and applications. Transl Res. 2009;154(6):277–87.CrossRef
9.
go back to reference Kennedy EH, Wiitala WL, Hayward RA, Sussman JB. Improved cardiovascular risk prediction using nonparametric regression and electronic health record data. Med Care. 2013;51(3):251.CrossRef Kennedy EH, Wiitala WL, Hayward RA, Sussman JB. Improved cardiovascular risk prediction using nonparametric regression and electronic health record data. Med Care. 2013;51(3):251.CrossRef
10.
go back to reference Collins GS, Mallett S, Omar O, Yu L-M. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011;9(1):103.CrossRef Collins GS, Mallett S, Omar O, Yu L-M. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011;9(1):103.CrossRef
11.
go back to reference Kahn HS, Cheng YJ, Thompson TJ, Imperatore G, Gregg EW. Two risk-scoring systems for predicting incident diabetes mellitus in US adults age 45 to 64 years. Ann Intern Med. 2009;150(11):741–51.CrossRef Kahn HS, Cheng YJ, Thompson TJ, Imperatore G, Gregg EW. Two risk-scoring systems for predicting incident diabetes mellitus in US adults age 45 to 64 years. Ann Intern Med. 2009;150(11):741–51.CrossRef
12.
go back to reference Tsiatis AA, Davidian M. Joint modeling of longitudinal and time-to-event data: an overview. Stat Sin. 2004;14(3):809–34. Tsiatis AA, Davidian M. Joint modeling of longitudinal and time-to-event data: an overview. Stat Sin. 2004;14(3):809–34.
13.
go back to reference Sweeting MJ, Thompson SG. Joint modelling of longitudinal and time-to-event data with application to predicting abdominal aortic aneurysm growth and rupture. Biom J. 2011;53(5):750–63.CrossRef Sweeting MJ, Thompson SG. Joint modelling of longitudinal and time-to-event data with application to predicting abdominal aortic aneurysm growth and rupture. Biom J. 2011;53(5):750–63.CrossRef
14.
go back to reference Guo X, Carlin BP. Separate and joint modeling of longitudinal and event time data using standard computer packages. Am Stat. 2004;58(1):16–24.CrossRef Guo X, Carlin BP. Separate and joint modeling of longitudinal and event time data using standard computer packages. Am Stat. 2004;58(1):16–24.CrossRef
15.
go back to reference Andrinopoulou ER, Eilers PHC, Takkenberg JJM, Rizopoulos D. Improved dynamic predictions from joint models of longitudinal and survival data with time-varying effects using P-splines. Biometrics. 2017;74(2):685–93.CrossRef Andrinopoulou ER, Eilers PHC, Takkenberg JJM, Rizopoulos D. Improved dynamic predictions from joint models of longitudinal and survival data with time-varying effects using P-splines. Biometrics. 2017;74(2):685–93.CrossRef
16.
go back to reference Njagi EN, Rizopoulos D, Molenberghs G, Dendale P, & Willekens K. A joint survival-longitudinal modelling approach for the dynamic prediction of rehospitalization in telemonitored chronic heart failure patients. Stat Model. 2013;13(3);179–98. Njagi EN, Rizopoulos D, Molenberghs G, Dendale P, & Willekens K. A joint survival-longitudinal modelling approach for the dynamic prediction of rehospitalization in telemonitored chronic heart failure patients. Stat Model. 2013;13(3);179–98.
17.
go back to reference Van Houwelingen H, Putter H. Dynamic prediction in clinical survival analysis. Boca Raton: CRC Press; 2011. Van Houwelingen H, Putter H. Dynamic prediction in clinical survival analysis. Boca Raton: CRC Press; 2011.
18.
go back to reference Van Houwelingen HC. Dynamic prediction by landmarking in event history analysis. Scand J Stat. 2007;34(1):70–85.CrossRef Van Houwelingen HC. Dynamic prediction by landmarking in event history analysis. Scand J Stat. 2007;34(1):70–85.CrossRef
19.
go back to reference Yokota I, Matsuyama Y. Dynamic prediction of repeated events data based on landmarking model: application to colorectal liver metastases data. BMC Med Res Methodol. 2019;19:31):1–11.CrossRef Yokota I, Matsuyama Y. Dynamic prediction of repeated events data based on landmarking model: application to colorectal liver metastases data. BMC Med Res Methodol. 2019;19:31):1–11.CrossRef
20.
go back to reference Cox DR. Regression models and life-tables. J R Stat Soc Ser B Methodol. 1972;34(2):187–202. Cox DR. Regression models and life-tables. J R Stat Soc Ser B Methodol. 1972;34(2):187–202.
21.
go back to reference Fleming TR, Harrington DP. Counting processes and survival analysis. Hoboken: Wiley; 2011. Fleming TR, Harrington DP. Counting processes and survival analysis. Hoboken: Wiley; 2011.
23.
go back to reference Kengne AP, Beulens JWJ, Peelen LM, Moons KGM, van der Schouw YT, Schulze MB, et al. Non-invasive risk scores for prediction of type 2 diabetes (EPIC-InterAct): a validation of existing models. Lancet Diabetes Endocrinol. 2014;2(1):19–29.CrossRef Kengne AP, Beulens JWJ, Peelen LM, Moons KGM, van der Schouw YT, Schulze MB, et al. Non-invasive risk scores for prediction of type 2 diabetes (EPIC-InterAct): a validation of existing models. Lancet Diabetes Endocrinol. 2014;2(1):19–29.CrossRef
24.
go back to reference Parast L, Cai T. Landmark risk prediction of residual life for breast cancer survival. Stat Med. 2013;32(20):3459–71.CrossRef Parast L, Cai T. Landmark risk prediction of residual life for breast cancer survival. Stat Med. 2013;32(20):3459–71.CrossRef
25.
go back to reference Parast L, Cheng SC, Cai T. Incorporating short-term outcome information to predict long-term survival with discrete markers. Biom J. 2011;53(2):294–307.CrossRef Parast L, Cheng SC, Cai T. Incorporating short-term outcome information to predict long-term survival with discrete markers. Biom J. 2011;53(2):294–307.CrossRef
26.
go back to reference Kalbfleisch JD, Prentice RL. The statistical analysis of failure time data. Hoboken: Wiley; 2011. Kalbfleisch JD, Prentice RL. The statistical analysis of failure time data. Hoboken: Wiley; 2011.
27.
go back to reference Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.CrossRef Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.CrossRef
28.
go back to reference Heagerty PJ, Zheng Y. Survival model predictive accuracy and ROC curves. Biometrics. 2005;61(1):92–105.CrossRef Heagerty PJ, Zheng Y. Survival model predictive accuracy and ROC curves. Biometrics. 2005;61(1):92–105.CrossRef
29.
go back to reference Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev. 1950;78(1):1–3.CrossRef Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev. 1950;78(1):1–3.CrossRef
30.
go back to reference Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. Cham: Springer Science & Business Media; 2008. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. Cham: Springer Science & Business Media; 2008.
31.
go back to reference Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med. 2015;34(10):1659–80.CrossRef Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med. 2015;34(10):1659–80.CrossRef
32.
go back to reference D'agostino R, Nam B-H. Evaluation of the performance of survival analysis models: discrimination and calibration measures. Handbook Statist. 2003;23:1–25.CrossRef D'agostino R, Nam B-H. Evaluation of the performance of survival analysis models: discrimination and calibration measures. Handbook Statist. 2003;23:1–25.CrossRef
33.
go back to reference Pencina MJ, D’Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72.CrossRef Pencina MJ, D’Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72.CrossRef
34.
go back to reference Pencina MJ, D'Agostino RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21.CrossRef Pencina MJ, D'Agostino RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21.CrossRef
35.
go back to reference Parast L, Cheng S-C, Cai T. Landmark prediction of long term survival incorporating short term event time information. J Am Stat Assoc. 2012;107(500):1492–501.CrossRef Parast L, Cheng S-C, Cai T. Landmark prediction of long term survival incorporating short term event time information. J Am Stat Assoc. 2012;107(500):1492–501.CrossRef
36.
go back to reference Efron B, Tibshirani RJ. An introduction to the bootstrap. Boca Raton: CRC Press; 1994. Efron B, Tibshirani RJ. An introduction to the bootstrap. Boca Raton: CRC Press; 1994.
37.
go back to reference Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc. 1983;78(382):316–31.CrossRef Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc. 1983;78(382):316–31.CrossRef
38.
go back to reference Picard RR, Cook RD. Cross-validation of regression models. J Am Stat Assoc. 1984;79(387):575–83.CrossRef Picard RR, Cook RD. Cross-validation of regression models. J Am Stat Assoc. 1984;79(387):575–83.CrossRef
39.
go back to reference American Diabetes A. The Diabetes prevention program. Design and methods for a clinical trial in the prevention of type 2 diabetes. Diabetes Care. 1999;22(4):623–34.CrossRef American Diabetes A. The Diabetes prevention program. Design and methods for a clinical trial in the prevention of type 2 diabetes. Diabetes Care. 1999;22(4):623–34.CrossRef
40.
go back to reference Diabetes Prevention Program Research G. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002;2002(346):393–403. Diabetes Prevention Program Research G. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002;2002(346):393–403.
41.
go back to reference Hostalek U, Gwilt M, Hildemann S. Therapeutic use of metformin in prediabetes and diabetes prevention. Drugs. 2015;75(10):1071–94.CrossRef Hostalek U, Gwilt M, Hildemann S. Therapeutic use of metformin in prediabetes and diabetes prevention. Drugs. 2015;75(10):1071–94.CrossRef
42.
go back to reference Diabetes Prevention Program Research G. The Diabetes prevention program: baseline characteristics of the randomized cohort. Diabetes Care. 2000;23(11):1619.CrossRef Diabetes Prevention Program Research G. The Diabetes prevention program: baseline characteristics of the randomized cohort. Diabetes Care. 2000;23(11):1619.CrossRef
43.
go back to reference Gail MH, Pfeiffer RM. On criteria for evaluating models of absolute risk. Biostatistics. 2005;6(2):227–39.CrossRef Gail MH, Pfeiffer RM. On criteria for evaluating models of absolute risk. Biostatistics. 2005;6(2):227–39.CrossRef
44.
go back to reference Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928–35.CrossRef Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928–35.CrossRef
45.
go back to reference Dale O, Hagen KB. Despite technical problems personal digital assistants outperform pen and paper when collecting patient diary data. J Clin Epidemiol. 2007;60(1):8–17.CrossRef Dale O, Hagen KB. Despite technical problems personal digital assistants outperform pen and paper when collecting patient diary data. J Clin Epidemiol. 2007;60(1):8–17.CrossRef
46.
go back to reference Wilcox AB, Gallagher KD, Boden-Albala B, Bakken SR. Research data collection methods: from paper to tablet computers. Med Care. 2012;50:S68–73.CrossRef Wilcox AB, Gallagher KD, Boden-Albala B, Bakken SR. Research data collection methods: from paper to tablet computers. Med Care. 2012;50:S68–73.CrossRef
47.
go back to reference Lloyd-Jones DM, Huffman MD, Karmali KN, Sanghavi DM, Wright JS, Pelser C, et al. Estimating longitudinal risks and benefits from cardiovascular preventive therapies among medicare patients. J Am Coll Cardiol. 2017;69(12):1617–36.CrossRef Lloyd-Jones DM, Huffman MD, Karmali KN, Sanghavi DM, Wright JS, Pelser C, et al. Estimating longitudinal risks and benefits from cardiovascular preventive therapies among medicare patients. J Am Coll Cardiol. 2017;69(12):1617–36.CrossRef
49.
go back to reference Hothorn T, Bühlmann P, Dudoit S, Molinaro A, Van Der Laan MJ. Survival ensembles. Biostatistics. 2005;7(3):355–73.CrossRef Hothorn T, Bühlmann P, Dudoit S, Molinaro A, Van Der Laan MJ. Survival ensembles. Biostatistics. 2005;7(3):355–73.CrossRef
50.
go back to reference Ridgeway G. The state of boosting. Comput Sci Stat. 1999:172–81. Ridgeway G. The state of boosting. Comput Sci Stat. 1999:172–81.
51.
go back to reference Burke HB, Goodman PH, Rosen DB, Henson DE, Weinstein JN, Harrell FE Jr, et al. Artificial neural networks improve the accuracy of cancer survival prediction. Cancer. 1997;79(4):857–62.CrossRef Burke HB, Goodman PH, Rosen DB, Henson DE, Weinstein JN, Harrell FE Jr, et al. Artificial neural networks improve the accuracy of cancer survival prediction. Cancer. 1997;79(4):857–62.CrossRef
52.
go back to reference Kappen HJ, Neijt JP. Neural network analysis to predict treatment outcome. Ann Oncol. 1993;4(suppl_4):S31–S4.CrossRef Kappen HJ, Neijt JP. Neural network analysis to predict treatment outcome. Ann Oncol. 1993;4(suppl_4):S31–S4.CrossRef
Metadata
Title
Dynamic risk prediction for diabetes using biomarker change measurements
Authors
Layla Parast
Megan Mathews
Mark W. Friedberg
Publication date
01-12-2019
Publisher
BioMed Central
Keyword
Metformin
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0812-y

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue