Skip to main content
Top
Published in: Drugs 4/2021

01-03-2021 | Type 2 Diabetes | Original Research Article

Personalized Multimorbidity Management for Patients with Type 2 Diabetes Using Reinforcement Learning of Electronic Health Records

Authors: Hua Zheng, Ilya O. Ryzhov, Wei Xie, Judy Zhong

Published in: Drugs | Issue 4/2021

Login to get access

Abstract

Background

Comorbid chronic conditions are common among people with type 2 diabetes. We developed an artificial intelligence algorithm, based on reinforcement learning (RL), for personalized diabetes and multimorbidity management, with strong potential to improve health outcomes relative to current clinical practice.

Methods

We modeled glycemia, blood pressure, and cardiovascular disease (CVD) risk as health outcomes, using a retrospective cohort of 16,665 patients with type 2 diabetes from New York University Langone Health ambulatory care electronic health records in 2009–2017. We trained an RL prescription algorithm that recommends a treatment regimen optimizing patients’ cumulative health outcomes using their individual characteristics and medical history at each encounter. The RL recommendations were evaluated on an independent subset of patients.

Results

The single-outcome optimization RL algorithms, RL–glycemia, RL–blood pressure, and RL–CVD, recommended consistent prescriptions as that observed by clinicians in 86.1%, 82.9%, and 98.4% of the encounters, respectively. For patient encounters in which the RL recommendations differed from the clinician prescriptions, significantly fewer encounters showed uncontrolled glycemia (A1c > 8% in 35% of encounters), uncontrolled hypertension (blood pressure > 140 mmHg in 16% of encounters), and high CVD risk (risk > 20% in 25% of encounters) under RL algorithms compared with those observed under clinicians (43%, 27%, and 31% of encounters, respectively; all p < 0.001).

Conclusions

A personalized RL prescriptive framework for type 2 diabetes yielded high concordance with clinicians’ prescriptions, and substantial improvements in glycemia, blood pressure, and CVD risk outcomes.
Appendix
Available only for authorised users
Literature
12.
go back to reference Ernst D, Stan G, Goncalves J, Wehenkel L, editors. Clinical data based optimal STI strategies for HIV: a reinforcement learning approach. Proceedings of the 45th IEEE Conference on Decision and Control; 13–15 Dec 2006: San Diego, CA. Ernst D, Stan G, Goncalves J, Wehenkel L, editors. Clinical data based optimal STI strategies for HIV: a reinforcement learning approach. Proceedings of the 45th IEEE Conference on Decision and Control; 13–15 Dec 2006: San Diego, CA.
13.
go back to reference Zhao Y, Zeng D, Socinski MA, Kosorok MR. Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer. Biometrics. 2011;67(4):1422–33.CrossRef Zhao Y, Zeng D, Socinski MA, Kosorok MR. Reinforcement learning strategies for clinical trials in nonsmall cell lung cancer. Biometrics. 2011;67(4):1422–33.CrossRef
19.
go back to reference Sutton RS, Barto AG. Reinforcement learning: an introduction. 2nd ed. Cambridge: The MIT Press; 2018. Sutton RS, Barto AG. Reinforcement learning: an introduction. 2nd ed. Cambridge: The MIT Press; 2018.
20.
go back to reference Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D et al. Playing atari with deep reinforcement learning. INIPS Deep Learning Workshop; 2013. Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D et al. Playing atari with deep reinforcement learning. INIPS Deep Learning Workshop; 2013.
23.
go back to reference Imbens G, Rubin DB. Causal inference for statistics, social, and biomedical sciences: an introduction. Cambridge: Cambridge University Press; 2015.CrossRef Imbens G, Rubin DB. Causal inference for statistics, social, and biomedical sciences: an introduction. Cambridge: Cambridge University Press; 2015.CrossRef
24.
go back to reference Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems; Long Beach: Curran Associates Inc.; 2017. pp. 4768–77. Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems; Long Beach: Curran Associates Inc.; 2017. pp. 4768–77.
28.
go back to reference Mataric MJ. Reward functions for accelerated learning. Machine learning proceedings 1994. Amsterdam: Elsevier; 1994. p. 181–9.CrossRef Mataric MJ. Reward functions for accelerated learning. Machine learning proceedings 1994. Amsterdam: Elsevier; 1994. p. 181–9.CrossRef
Metadata
Title
Personalized Multimorbidity Management for Patients with Type 2 Diabetes Using Reinforcement Learning of Electronic Health Records
Authors
Hua Zheng
Ilya O. Ryzhov
Wei Xie
Judy Zhong
Publication date
01-03-2021
Publisher
Springer International Publishing
Keyword
Type 2 Diabetes
Published in
Drugs / Issue 4/2021
Print ISSN: 0012-6667
Electronic ISSN: 1179-1950
DOI
https://doi.org/10.1007/s40265-020-01435-4

Other articles of this Issue 4/2021

Drugs 4/2021 Go to the issue