Top

BMC Medical Informatics and Decision Making

Published in:

Open Access 01-12-2020 | Myocardial Infarction | Research article

A stacking-based model for predicting 30-day all-cause hospital readmissions of patients with acute myocardial infarction

Authors: Zhen Zhang, Hang Qiu, Weihao Li, Yucheng Chen

Published in: BMC Medical Informatics and Decision Making | Issue 1/2020

Abstract

Background

Acute myocardial infarction (AMI) is a serious cardiovascular disease, followed by a high readmission rate within 30-days of discharge. Accurate prediction of AMI readmission is a crucial way to identify the high-risk group and optimize the distribution of medical resources.

Methods

In this study, we propose a stacking-based model to predict the risk of 30-day unplanned all-cause hospital readmissions for AMI patients based on clinical data. Firstly, we conducted an under-sampling method of neighborhood cleaning rule (NCR) to alleviate the class imbalance and then utilized a feature selection method of SelectFromModel (SFM) to select effective features. Secondly, we adopted a self-adaptive approach to select base classifiers from eight candidate models according to their performances in datasets. Finally, we constructed a three-layer stacking model in which layer 1 and layer 2 were base-layer and level 3 was meta-layer. The predictions of the base-layer were used to train the meta-layer in order to make the final forecast.

Results

The results show that the proposed model exhibits the highest AUC (0.720), which is higher than that of decision tree (0.681), support vector machine (0.707), random forest (0.701), extra trees (0.709), adaBoost (0.702), bootstrap aggregating (0.704), gradient boosting decision tree (0.710) and extreme gradient enhancement (0.713).

Conclusion

It is evident that our model could effectively predict the risk of 30-day all cause hospital readmissions for AMI patients and provide decision support for the administration.

Available only for authorised users

Reed GW, Rossi JE, Cannon CP. Acute myocardial infarction. Lancet. 2017;389:197–210. https://doi.org/10.1016/S0140-6736(16)30677-8.CrossRefPubMed

Fingar K, Washington R. Trends in Hospital Readmissions for Four High-Volume Conditions, 2009–2013: Statistical Brief #196. In: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville: Agency for Healthcare Research and Quality (US); 2006. http://www.ncbi.nlm.nih.gov/books/NBK338299/. Accessed 8 May 2020.

Umscheid CA, Behta M, VanZandbergen C, Tait G, Hanish A, William Hanson C, et al. The readmission risk flag: using the electronic health record to automatically identify patients at risk for 30-day readmission. J Hosp Med. 2013;8. https://doi.org/10.1002/jhm.2106.

Amarasingham R, Moore BJ, Tabak YP, Drazner MH, Clark CA, Zhang S, et al. An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Med Care. 2010;48:981–8. https://doi.org/10.1097/mlr.0b013e3181ef60d9.CrossRefPubMed

Amarasingham R, Patel PC, Toto K. Allocating scarce resources in real-time to reduce heart failure readmissions: a prospective, controlled study. BMJ Qual Saf. 2013. https://doi.org/10.1136/bmjqs-2013-001901.

Van Walraven C, Dhalla IA, Bell C, Etchells E, Stiell IG, Zarnke K, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. Cmaj Can Med Assoc J. 2010;182:551–7. https://doi.org/10.1503/cmaj.091117.CrossRef

Cotter PE, Bhalla VK, Wallis SJ, Biram RWS. Predicting readmissions: poor performance of the LACE index in an older UK population. Age Ageing. 2012;(6). https://doi.org/10.1093/ageing/afs073.

Artetxe A, Beristain A, Graña M. Predictive models for hospital readmission risk: a systematic review of methods. Comput Methods Prog Biomed. 2018;164:49–64. https://doi.org/10.1016/j.cmpb.2018.06.006.CrossRef

Tan B-Y, Gu J-Y, Wei H-Y, Chen L, Yan S-L, Deng N. Electronic medical record-based model to predict the risk of 90-day readmission for patients with heart failure. BMC Med Inform Decis Making. 2019;19:193.CrossRef

10.

Awan SE, Bennamoun M, Sohel F, Sanfilippo FM, Dwivedi G. Machine learning-based prediction of heart failure readmission or death: implications of choosing the right model and the right metrics. ESC Heart Failure. 2019;6:428–35.CrossRef

11.

Cui S, Wang D, Wang Y, Yu P-W, Jin Y. An improved support vector machine-based diabetic readmission prediction. Comput Methods Prog Biomed. 2018;166:123–35. https://doi.org/10.1016/j.cmpb.2018.10.012.CrossRef

12.

Golas SB, Shibahara T, Agboola S, Otaki H, Sato J, Nakae T, et al. A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak. 2018;18:44.CrossRef

13.

Mortazavi B, Downing N, Bucholz E, Dharmarajan K, Manhapra A, Li S, et al. Analysis of machine learning techniques for heart failure readmissions. Circ Cardiovasc Qual Outcomes. 2016;9:629–40. https://doi.org/10.1161/CIRCOUTCOMES.116.003039.CrossRefPubMedPubMedCentral

14.

Walsh C, Hripcsak G. The effects of data sources, cohort selection, and outcome definition on a predictive model of risk of thirty-day hospital readmissions. J Biomed Inform. 2014;52:418–26. https://doi.org/10.1016/j.jbi.2014.08.006.CrossRefPubMedPubMedCentral

15.

Gupta S, Ko DT, Azizi P, Bouadjenek MR, Koh M, Chong A, et al. Evaluation of machine learning algorithms for predicting readmission after acute myocardial infarction using routinely collected clinical data. Can J Cardiol. 2019. https://doi.org/10.1016/j.cjca.2019.10.023.

16.

Breiman L. Bagging predictors. Mach Learn. 1996;24:123–40.

17.

Freund Y, Schapire RE. Experiments with a new boosting algorithm. Experiments with a new boosting algorithm; 1996. p. 148–56.

18.

Wolpert DH. Stacked generalization. Neural Netw. 1992;5:241–59. https://doi.org/10.1016/s0893-6080(05)80023-1.CrossRef

19.

Radovanović S, Delibašić B, Jovanović M, Vukićević M, Suknović M. Framework for integration of domain knowledge into logistic regression. In: Proceedings of the 8th international conference on web intelligence, mining and semantics. Novi Sad: Association for Computing Machinery; 2018. p. 1–8. https://doi.org/10.1145/3227609.3227653.CrossRef

20.

Yu K, Xie X. Predicting hospital readmission: a joint ensemble-learning model. IEEE J Biomed Health Inform. 2020;24:447–56.CrossRef

21.

Li X, Liu H, Yang J, Xie G, Xu M, Yang Y. Using machine learning models to predict in-hospital mortality for ST-elevation myocardial infarction patients. Stud Health Technol Inform. 2017;245:476–80.PubMed

22.

Pan L, Liu G, Lin F, Zhong S, Xia H, Sun X, et al. Machine learning applications for prediction of relapse in childhood acute lymphoblastic leukemia. Sci Rep. 2017;7. https://doi.org/10.1038/s41598-017-07408-0.

23.

Li DC, Liu CW, Hu SC. A learning method for the class imbalance problem with medical data sets. Comput Biol Med. 2010;40:509–18. https://doi.org/10.1016/j.compbiomed.2010.03.005.CrossRefPubMed

24.

Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program | NEJM. N Engl J Med. 2009;361:1418–28. https://doi.org/10.1056/NEJMc090911.CrossRef

25.

Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res. 2002;16:321–57. https://doi.org/10.1613/jair.953.CrossRef

26.

Elkan C. The Foundation of Cost-sensitive Learning. In: Proc.seventeenth Intl.joint Conf.on Artificial Intelligence; 2001. http://ci.nii.ac.jp/naid/10012857761. Accessed 23 Aug 2020.

27.

Laurikkala J. Improving identification of difficult small classes by balancing class distribution. In: Conference on Ai in Medicine in Europe: Artificial Intelligence Medicine; 2001. https://doi.org/10.1007/3-540-48229-6_9.CrossRef

28.

Vincent DR, Mahendran N. Effective classification of major depressive disorder patients using machine learning techniques. Recent Pat Comput Sci. 2018;11. https://doi.org/10.2174/2213275911666181016160920.

29.

Rodriguez-Galiano VF, Luque-Espinar JA, Chica-Olmo M, Mendes MP. Feature selection approaches for predictive modelling of groundwater nitrate pollution: an evaluation of filters, embedded and wrapper methods. Sci Total Environ. 2018;624:661–72. https://doi.org/10.1016/j.scitotenv.2017.12.152.CrossRefPubMed

30.

Saeys Y, Inza I, Larrañaga P. A review of feature selection techniques in bioinformatics. Bioinformatics. 2007;23:2507–17. https://doi.org/10.1093/bioinformatics/btm344.CrossRefPubMed

31.

Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International joint conference on Artificial intelligence; 1995. http://www.researchgate.net/publication/2352264_A_Study_of_Cross-Validation_and_Bootstrap_for_Accuracy_Estimation_and_Model_Selection. Accessed 4 May 2020.

32.

Singh M, Singh P, Singh H. Decision tree classifier for human protein function prediction. In: International Conference on Advanced Computing & Communications; 2006. https://doi.org/10.1109/ADCOM.2006.4289955.CrossRef

33.

Danenas P, Garsva G. Credit risk evaluation modeling using evolutionary linear SVM classifiers and sliding window approach. Proc Comput Sci. 2012;9:1324–33. https://doi.org/10.1016/j.procs.2012.04.145.CrossRef

34.

Waljee AK, Liu B, Sauder K, Zhu J, Govani SM, Stidham RW, et al. Predicting corticosteroid-free endoscopic remission with vedolizumab in ulcerative colitis. Aliment Pharmacol Ther. 2018. https://doi.org/10.1111/apt.14510.

35.

Geurts P, Ernst D, Wehenkel L. Extremely randomized trees. Mach Learn. 2006;63:3–42. https://doi.org/10.1007/s10994-006-6226-1.CrossRef

36.

Rätsch G, Onoda T, Müller KR. Soft Margins for AdaBoost. Mach Learn. 2001;42:287–320. https://doi.org/10.1023/A:1007618119488.CrossRef

37.

Dong L, Yuan Y, Cai Y. Using bagging classifier to predict protein domain structural class. J Biomol Struct Dyn. 2007;24:239–42. https://doi.org/10.1007/s10858-006-9062-5.CrossRef

38.

Niu F, Recht B, Re C. HOGWILD!: a lock-free approach to parallelizing stochastic gradient descent. Adv Neural Inf Proces Syst. 2011; http://www.researchgate.net/publication/51951195_HOGWILD!_A_Lock-Free_Approach_to_Parallelizing_Stochastic_GradientDescent. Accessed 4 May 2020.

39.

Chen T, Tong H, Benesty M. xgboost: Extreme Gradient Boosting. 2016. http://star-www.st-andrews.ac.uk/cran/web/packages/xgboost/index.html. Accessed 4 May 2020.CrossRef

40.

Krishnapuram B, Carin L, Figueiredo MA, Hartemink AJ. Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Trans Pattern Anal Mach Intell. 2005;27:957. https://doi.org/10.1109/TPAMI.2005.127.CrossRefPubMed

41.

Tang J, Liang J, Han C, Li Z, Huang H. Crash injury severity analysis using a two-layer stacking framework. Accid Anal Prev. 2019;122:226–38. https://doi.org/10.1016/j.aap.2018.10.016.CrossRefPubMed

42.

Steyerberg EW, Vickers AJ, Cook NR, Gerds TA, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–38. https://doi.org/10.1097/EDE.0b013e3181c30fb2.CrossRefPubMedPubMedCentral

43.

Xu Y, Yang X, Huang H, Peng C, Ge Y, Wu H, et al. Extreme gradient boosting model has a better performance in predicting the risk of 90-day readmissions in patients with Ischaemic stroke. J Stroke Cerebrovasc Dis. 2019;28:104441. https://doi.org/10.1016/j.jstrokecerebrovasdis.2019.104441.CrossRefPubMed

44.

Morel D, Yu KC, Liu-Ferrara A, Caceres-Suriel AJ, Kurtz SG, Tabak YP. Predicting hospital readmission in patients with mental or substance use disorders: a machine learning approach. Int J Med Inform. 2020;139:104136.CrossRef

45.

Turgeman L, May JH. A mixed-ensemble model for hospital readmission. Artif Intell Med. 2016;72:72–82. https://doi.org/10.1016/j.artmed.2016.08.005.CrossRefPubMed

46.

Krumholz HM, Lin Z, Keenan PS, Chen J, Ross JS, Drye EE, et al. Relationship between hospital readmission and mortality rates for patients hospitalized with acute myocardial infarction, heart failure, or pneumonia. Jama J Am Med Assoc. 2013;309:587. https://doi.org/10.1001/jama.2013.333.CrossRef

47.

Khera R, Jain S, Pandey A, Agusala V, Kumbhani DJ, Das SR, et al. Comparison of Readmission Rates After Acute Myocardial Infarction in 3 Patient Age Groups (18 to 44, 45 to 64, and ≥65 Years) in the United States. Am J Cardiol. 2017:S0002914917312870. https://doi.org/10.1016/j.amjcard.2017.07.081.

48.

Smith Lauren N, Makam Anil N, Douglas D, Helen M, Das Sandeep R, Halm Ethan A, et al. Acute myocardial infarction readmission risk prediction models. Circulation. 2018;11:e003885. https://doi.org/10.1161/CIRCOUTCOMES.117.003885.CrossRefPubMed

49.

Kwok CS, Wong CW, Shufflebotham H, Brindley L, Fatima T, Shufflebotham A, et al. Early readmissions after acute myocardial infarction. Am J Cardiol. 2017;120:723–8. https://doi.org/10.1016/j.amjcard.2017.05.049.CrossRefPubMed

50.

Shams I, Ajorlou S, Yang K. A predictive analytics approach to reducing 30-day avoidable readmissions among patients with heart failure, acute myocardial infarction, pneumonia, or COPD. Health Care Manag Sci. 2015;18:19–34. https://doi.org/10.1007/s10729-014-9278-y.CrossRefPubMed

51.

Asche CV, Ren J, Kirkness CS, Kim M, Dong Y, Hippler S. A prediction model to identify acute myocardial infarction (AMI) patients at risk for 30-day readmission. In: Proceedings of the Summer Computer simulation conference. Montreal: Society for Computer Simulation International; 2016. p. 1–8.

52.

Krumholz HM, Lin Z, Drye EE, Desai MM, Han LF, Rapp MT, et al. An administrative claims measure suitable for profiling hospital performance based on 30-day all-cause readmission rates among patients with acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2011;4:243–52.CrossRef

53.

Yu S, Farooq F, van Esbroeck A, Fung G, Anand V, Krishnapuram B. Predicting readmission risk with institution-specific prediction models. Artif Intell Med. 2015;65:89–96. https://doi.org/10.1016/j.artmed.2015.08.005.CrossRefPubMed

54.

Tabachnick BG, Fidell LS. Using Multivariate Statistics (5th Ed.): Pearson/Allyn & Bacon; 2007. https://doi.org/10.1021/jm000292e.

55.

Stephanie R, Oanh K, Nguyen M, et al. Hospital-initiated transitional care interventions as a patient safety strategy: a systematic review. Ann Intern Med. 2013. https://doi.org/10.7326/0003-4819-158-5-201303051-00011.

Title: A stacking-based model for predicting 30-day all-cause hospital readmissions of patients with acute myocardial infarction
Authors: Zhen Zhang
Hang Qiu
Weihao Li
Yucheng Chen
Publication date: 01-12-2020
Publisher: BioMed Central
Keyword: Myocardial Infarction
Published in: BMC Medical Informatics and Decision Making / Issue 1/2020
Electronic ISSN: 1472-6947
DOI: https://doi.org/10.1186/s12911-020-01358-w

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

A stacking-based model for predicting 30-day all-cause hospital readmissions of patients with acute myocardial infarction

Abstract

Background

Methods

Results

Conclusion

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2020

Home blood pressure data visualization for the management of hypertension: designing for patient and physician information needs

Can natural language processing help differentiate inflammatory intestinal diseases in China? Models applying random forest and convolutional neural network approaches

Diagnosis of left ventricular hypertrophy using convolutional neural network

Investigating the satisfaction level of physicians in regards to implementing medical Picture Archiving and Communication System (PACS)

Design and clinical validation of a software program for automated measurement of mammographic breast density

ICT-based adherence monitoring in kidney transplant recipients: a randomized controlled trial