Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2022

Open Access 01-12-2022 | Angiography | Research

A machine learning approach for modeling decisions in the out of hospital cardiac arrest care workflow

Authors: Samuel Harford, Marina Del Rios, Sara Heinert, Joseph Weber, Eddie Markul, Katie Tataris, Teri Campbell, Terry Vanden Hoek, Houshang Darabi

Published in: BMC Medical Informatics and Decision Making | Issue 1/2022

Login to get access

Abstract

Background

A growing body of research has shown that machine learning (ML) can be a useful tool to predict how different variable combinations affect out-of-hospital cardiac arrest (OHCA) survival outcomes. However, there remain significant research gaps on the utilization of ML models for decision-making and their impact on survival outcomes. The purpose of this study was to develop ML models that effectively predict hospital’s practice to perform coronary angiography (CA) in adult patients after OHCA and subsequent neurologic outcomes.

Methods

We utilized all (N = 2398) patients treated by the Chicago Fire Department Emergency Medical Services included in the Cardiac Arrest Registry to Enhance Survival (CARES) between 2013 and 2018 who survived to hospital admission to develop, test, and analyze ML models for decisions after return of spontaneous circulation (ROSC) and patient survival. ML classification models, including the Embedded Fully Convolutional Network (EFCN) model, were compared based on their ability to predict post-ROSC decisions and survival.

Results

The EFCN classification model achieved the best results across tested ML algorithms. The area under the receiver operating characteristic curve (AUROC) for CA and Survival were 0.908 and 0.896 respectively. Through cohort analyses, our model predicts that 18.3% (CI 16.4–20.2) of patients should receive a CA that did not originally, and 30.1% (CI 28.5–31.7) of these would experience improved survival outcomes.

Conclusion

ML modeling effectively predicted hospital decisions and neurologic outcomes. ML modeling may serve as a quality improvement tool to inform system level OHCA policies and treatment protocols.
Appendix
Available only for authorised users
Literature
1.
go back to reference Daya MR, Schmicker RH, Zive DM, et al. Out-of-hospital cardiac arrest survival improving over time: results from the resuscitation outcomes consortium (ROC). Resuscitation. 2015;91:108–15.CrossRef Daya MR, Schmicker RH, Zive DM, et al. Out-of-hospital cardiac arrest survival improving over time: results from the resuscitation outcomes consortium (ROC). Resuscitation. 2015;91:108–15.CrossRef
2.
go back to reference Govindarajan P, Lin L, Landman A, McMullan JT, McNally BF, Crouch AJ, Sasson C. Practice variability among the EMS systems participating in cardiac arrest registry to enhance survival (CARES). Resuscitation. 2012;83(1):76–80.CrossRef Govindarajan P, Lin L, Landman A, McMullan JT, McNally BF, Crouch AJ, Sasson C. Practice variability among the EMS systems participating in cardiac arrest registry to enhance survival (CARES). Resuscitation. 2012;83(1):76–80.CrossRef
4.
go back to reference Wiens J, Shenoy ES. Machine learning for healthcare: on the verge of a major shift in healthcare epidemiology. Clin Infect Dis. 2018;66(1):149–53.CrossRef Wiens J, Shenoy ES. Machine learning for healthcare: on the verge of a major shift in healthcare epidemiology. Clin Infect Dis. 2018;66(1):149–53.CrossRef
5.
go back to reference Ngiam KY, Khor IW. Big data and machine learning algorithms for health-care delivery. Lancet Oncol. 2019;20(5):e262–73.CrossRef Ngiam KY, Khor IW. Big data and machine learning algorithms for health-care delivery. Lancet Oncol. 2019;20(5):e262–73.CrossRef
6.
go back to reference Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):1317–8.CrossRef Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):1317–8.CrossRef
7.
go back to reference Giger ML. Machine learning in medical imaging. J Am College Radiol. 2018;15(3):512–20.CrossRef Giger ML. Machine learning in medical imaging. J Am College Radiol. 2018;15(3):512–20.CrossRef
8.
go back to reference Ardabili SF, Mosavi A, Ghamisi P, Ferdinand F, Varkonyi-Koczy AR, Reuter U, Rabczuk T, Atkinson PM. Covid-19 outbreak prediction with machine learning. Available at SSRN 3580188. 2020. Ardabili SF, Mosavi A, Ghamisi P, Ferdinand F, Varkonyi-Koczy AR, Reuter U, Rabczuk T, Atkinson PM. Covid-19 outbreak prediction with machine learning. Available at SSRN 3580188. 2020.
9.
go back to reference Vamathevan J, Clark D, Czodrowski P, Dunham I, Ferran E, Lee G, Li B, Madabhushi A, Shah P, Spitzer M, Zhao S. Applications of machine learning in drug discovery and development. Nat Rev Drug Discov. 2019;18(6):463–77.CrossRef Vamathevan J, Clark D, Czodrowski P, Dunham I, Ferran E, Lee G, Li B, Madabhushi A, Shah P, Spitzer M, Zhao S. Applications of machine learning in drug discovery and development. Nat Rev Drug Discov. 2019;18(6):463–77.CrossRef
10.
go back to reference Sharabiani A, Darabi H, Bress A, Cavallari L, Nutescu E, Drozda K. Machine learning based prediction of warfarin optimal dosing for African American patients. In: 2013 IEEE international conference on automation science and engineering (CASE) 2013; pp 623–628. IEEE. Sharabiani A, Darabi H, Bress A, Cavallari L, Nutescu E, Drozda K. Machine learning based prediction of warfarin optimal dosing for African American patients. In: 2013 IEEE international conference on automation science and engineering (CASE) 2013; pp 623–628. IEEE.
11.
go back to reference Darabi H, Galanter WL, Lin JY, Buy U, Sampath R. Modeling and integration of hospital information systems with Petri nets. In: 2009 IEEE/INFORMS international conference on service operations, logistics and informatics 2009, pp 190–195. IEEE. Darabi H, Galanter WL, Lin JY, Buy U, Sampath R. Modeling and integration of hospital information systems with Petri nets. In: 2009 IEEE/INFORMS international conference on service operations, logistics and informatics 2009, pp 190–195. IEEE.
12.
go back to reference Haji M, Darabi H. A simulation case study: Reducing outpatient waiting time of otolaryngology care services using VBA. In: 2011 IEEE international conference on automation science and engineering 2011, pp 525–530. IEEE. Haji M, Darabi H. A simulation case study: Reducing outpatient waiting time of otolaryngology care services using VBA. In: 2011 IEEE international conference on automation science and engineering 2011, pp 525–530. IEEE.
13.
go back to reference Blomberg SN, Folke F, Ersbøll AK, Christensen HC, Torp-Pedersen C, Sayre MR, Counts CR, Lippert FK. Machine learning as a supportive tool to recognize cardiac arrest in emergency calls. Resuscitation. 2019;138:322–9.CrossRef Blomberg SN, Folke F, Ersbøll AK, Christensen HC, Torp-Pedersen C, Sayre MR, Counts CR, Lippert FK. Machine learning as a supportive tool to recognize cardiac arrest in emergency calls. Resuscitation. 2019;138:322–9.CrossRef
15.
go back to reference Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc. 2018;7(13):e008678.CrossRef Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc. 2018;7(13):e008678.CrossRef
16.
go back to reference Krizmaric M, Verlic M, Stiglic G, Grmec S, Kokol P. Intelligent analysis in predicting outcome of out-of-hospital cardiac arrest. Comput Methods Progr Biomed. 2009;95(2 Suppl):S22-32.CrossRef Krizmaric M, Verlic M, Stiglic G, Grmec S, Kokol P. Intelligent analysis in predicting outcome of out-of-hospital cardiac arrest. Comput Methods Progr Biomed. 2009;95(2 Suppl):S22-32.CrossRef
17.
go back to reference Nanayakkara S, Fogarty S, Tremeer M, et al. Characterising risk of in-hospital mortality following cardiac arrest using machine learning: a retrospective international registry study. PLoS Med. 2018;15(11):e1002709.CrossRef Nanayakkara S, Fogarty S, Tremeer M, et al. Characterising risk of in-hospital mortality following cardiac arrest using machine learning: a retrospective international registry study. PLoS Med. 2018;15(11):e1002709.CrossRef
21.
go back to reference McNally B, Stokes A, Crouch A, Kellermann AL, CARES Surveillance Group. CARES: cardiac arrest registry to enhance survival. Ann Emerg Med. 2009;54(5):674–83. McNally B, Stokes A, Crouch A, Kellermann AL, CARES Surveillance Group. CARES: cardiac arrest registry to enhance survival. Ann Emerg Med. 2009;54(5):674–83.
22.
go back to reference Potdar K, Pardawala TS, Pai CD. A comparative study of categorical variable encoding techniques for neural network classifiers. Int J Comput Appl. 2017;175(4):7–9. Potdar K, Pardawala TS, Pai CD. A comparative study of categorical variable encoding techniques for neural network classifiers. Int J Comput Appl. 2017;175(4):7–9.
23.
go back to reference Reynolds JC, Callaway CW, El Khoudary SR, Moore CG, Alvarez RJ, Rittenberger JC. Coronary angiography predicts improved outcome following cardiac arrest: propensity-adjusted analysis. J Intensive Care Med. 2009;24:179–86.CrossRef Reynolds JC, Callaway CW, El Khoudary SR, Moore CG, Alvarez RJ, Rittenberger JC. Coronary angiography predicts improved outcome following cardiac arrest: propensity-adjusted analysis. J Intensive Care Med. 2009;24:179–86.CrossRef
24.
go back to reference Dumas F, Cariou A, Manzo-Silberman S, Grimaldi D, Vivien B, Rosencher J, Empana JP, Carli P, Mira JP, Jouven X, Spaulding C. Immediate percutaneous coronary intervention is associated with better survival after out-of-hospital cardiac arrest: insights from the PROCAT (Parisian Region Out of hospital Cardiac ArresT) registry. Circul: Cardiovasc Interv. 2010;3(3):200–7. Dumas F, Cariou A, Manzo-Silberman S, Grimaldi D, Vivien B, Rosencher J, Empana JP, Carli P, Mira JP, Jouven X, Spaulding C. Immediate percutaneous coronary intervention is associated with better survival after out-of-hospital cardiac arrest: insights from the PROCAT (Parisian Region Out of hospital Cardiac ArresT) registry. Circul: Cardiovasc Interv. 2010;3(3):200–7.
25.
go back to reference Hollenbeck RD, McPherson JA, Mooney MR, Unger BT, Patel NC, McMullan PW Jr, Hsu CH, Seder DB, Kern KB. Early cardiac catheterization is associated with improved survival in comatose survivors of cardiac arrest without STEMI. Resuscitation. 2014;85:88–95.CrossRef Hollenbeck RD, McPherson JA, Mooney MR, Unger BT, Patel NC, McMullan PW Jr, Hsu CH, Seder DB, Kern KB. Early cardiac catheterization is associated with improved survival in comatose survivors of cardiac arrest without STEMI. Resuscitation. 2014;85:88–95.CrossRef
26.
go back to reference Grossestreuer AV, Abella BS, Sheak KR, et al. Inter-rater reliability of post-arrest cerebral performance category (CPC) scores. Resuscitation. 2016;109:21–4.CrossRef Grossestreuer AV, Abella BS, Sheak KR, et al. Inter-rater reliability of post-arrest cerebral performance category (CPC) scores. Resuscitation. 2016;109:21–4.CrossRef
27.
go back to reference Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY. Lightgbm: A highly efficient gradient boosting decision tree. In: Advances in neural information processing systems 2017, pp 3146–3154. Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY. Lightgbm: A highly efficient gradient boosting decision tree. In: Advances in neural information processing systems 2017, pp 3146–3154.
28.
go back to reference Chen T, Guestrin C. Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining 2016, pp 785–794. Chen T, Guestrin C. Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining 2016, pp 785–794.
29.
go back to reference Safavian SR, Landgrebe D. A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern. 1991;21(3):660–74.CrossRef Safavian SR, Landgrebe D. A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern. 1991;21(3):660–74.CrossRef
30.
go back to reference Liaw A, Wiener M. Classification and regression by random Forest. R news. 2002;2(3):18–22. Liaw A, Wiener M. Classification and regression by random Forest. R news. 2002;2(3):18–22.
31.
go back to reference Peter S, Diego F, Hamprecht FA, Nadler B. Cost efficient gradient boosting. In: Advances in neural information processing systems 2017, pp 1551–1561. Peter S, Diego F, Hamprecht FA, Nadler B. Cost efficient gradient boosting. In: Advances in neural information processing systems 2017, pp 1551–1561.
33.
go back to reference Hosmer DW Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. Hoboken: Wiley; 2013.CrossRef Hosmer DW Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. Hoboken: Wiley; 2013.CrossRef
34.
go back to reference Yu H, Kim S. SVM tutorial-classification, regression and ranking. Handb Natural Comput. 2012;1:479–506.CrossRef Yu H, Kim S. SVM tutorial-classification, regression and ranking. Handb Natural Comput. 2012;1:479–506.CrossRef
35.
go back to reference LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.CrossRef LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.CrossRef
36.
go back to reference Harford S, Darabi H, Del Rios M, Majumdar S, Karim F, Hoek TV, Erwin K, Watson DP. A machine learning based model for out of hospital cardiac arrest outcome classification and sensitivity analysis. Resuscitation. 2019;138:134–40.CrossRef Harford S, Darabi H, Del Rios M, Majumdar S, Karim F, Hoek TV, Erwin K, Watson DP. A machine learning based model for out of hospital cardiac arrest outcome classification and sensitivity analysis. Resuscitation. 2019;138:134–40.CrossRef
37.
go back to reference Jia Y, Zhang Y, Weiss R, Wang Q, Shen J, Ren F, Nguyen P, Pang R, Moreno IL, Wu Y. Transfer learning from speaker verification to multispeaker text-to-speech synthesis. In: Advances in neural information processing systems 2018, pp 4480–4490. Jia Y, Zhang Y, Weiss R, Wang Q, Shen J, Ren F, Nguyen P, Pang R, Moreno IL, Wu Y. Transfer learning from speaker verification to multispeaker text-to-speech synthesis. In: Advances in neural information processing systems 2018, pp 4480–4490.
38.
go back to reference Chicco D, Jurman G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics. 2020;21(1):1–3.CrossRef Chicco D, Jurman G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics. 2020;21(1):1–3.CrossRef
39.
go back to reference Roulston MS. Performance targets and the Brier score. Meteorol Appl: J Forecast Pract Appl Train Tech Model. 2007;14(2):185–94.CrossRef Roulston MS. Performance targets and the Brier score. Meteorol Appl: J Forecast Pract Appl Train Tech Model. 2007;14(2):185–94.CrossRef
40.
go back to reference Sundararajan M, Najmi A. The many Shapley values for model explanation. In: International conference on machine learning 2020, pp 9269–9278. PMLR. Sundararajan M, Najmi A. The many Shapley values for model explanation. In: International conference on machine learning 2020, pp 9269–9278. PMLR.
41.
go back to reference Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. 2019;380(14):1347–58.CrossRef Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. 2019;380(14):1347–58.CrossRef
42.
go back to reference Al-Dury N, Ravn-Fischer A, Hollenberg J, Israelsson J, Nordberg P, Strömsöe A, Axelsson C, Herlitz J, Rawshani A. Identifying the relative importance of predictors of survival in out of hospital cardiac arrest: a machine learning study. Scand J Trauma Resusc Emerg Med. 2020;28(1):1–8.CrossRef Al-Dury N, Ravn-Fischer A, Hollenberg J, Israelsson J, Nordberg P, Strömsöe A, Axelsson C, Herlitz J, Rawshani A. Identifying the relative importance of predictors of survival in out of hospital cardiac arrest: a machine learning study. Scand J Trauma Resusc Emerg Med. 2020;28(1):1–8.CrossRef
44.
go back to reference Callaway CW, Schmicker R, Kampmeyer M, et al. Receiving hospital characteristics associated with survival after out-of-hospital cardiac arrest. Resuscitation. 2010;81(5):524–9.CrossRef Callaway CW, Schmicker R, Kampmeyer M, et al. Receiving hospital characteristics associated with survival after out-of-hospital cardiac arrest. Resuscitation. 2010;81(5):524–9.CrossRef
45.
go back to reference Carr BG, Kahn JM, Merchant RM, et al. Inter-hospital variability in post-cardiac arrest mortality. Resuscitation. 2009;80(1):30–4.CrossRef Carr BG, Kahn JM, Merchant RM, et al. Inter-hospital variability in post-cardiac arrest mortality. Resuscitation. 2009;80(1):30–4.CrossRef
46.
go back to reference Schober A, Sterz F, Laggner AN, et al. Admission of out-of-hospital cardiac arrest victims to a high volume cardiac arrest center is linked to improved outcome. Resuscitation. 2016;106:42–8.CrossRef Schober A, Sterz F, Laggner AN, et al. Admission of out-of-hospital cardiac arrest victims to a high volume cardiac arrest center is linked to improved outcome. Resuscitation. 2016;106:42–8.CrossRef
47.
go back to reference Blum N, Del Rios M, Kotini P, Nguyen H, Campbell T, Markul E, Weber J, Vanden HT. Interhospital variability in out-of-hospital cardiac arrest treatment and survival in a large metropolitan Aea. Acad Emerg Med. 2019;26(S1):A353. Blum N, Del Rios M, Kotini P, Nguyen H, Campbell T, Markul E, Weber J, Vanden HT. Interhospital variability in out-of-hospital cardiac arrest treatment and survival in a large metropolitan Aea. Acad Emerg Med. 2019;26(S1):A353.
Metadata
Title
A machine learning approach for modeling decisions in the out of hospital cardiac arrest care workflow
Authors
Samuel Harford
Marina Del Rios
Sara Heinert
Joseph Weber
Eddie Markul
Katie Tataris
Teri Campbell
Terry Vanden Hoek
Houshang Darabi
Publication date
01-12-2022
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2022
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-021-01730-4

Other articles of this Issue 1/2022

BMC Medical Informatics and Decision Making 1/2022 Go to the issue