Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2023

Open Access 01-12-2023 | Research

Prediction of inappropriate pre-hospital transfer of patients with suspected cardiovascular emergency diseases using machine learning: a retrospective observational study

Authors: Ji Hoon Kim, Bomgyeol Kim, Min Joung Kim, Heejung Hyun, Hyeon Chang Kim, Hyuk-Jae Chang

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Login to get access

Abstract

Background

This study aimed to develop a prediction model for transferring patients to an inappropriate hospital for suspected cardiovascular emergency diseases at the pre-hospital stage, using variables obtained from an integrated nationwide dataset, and to assess the performance of this model.

Methods

We integrated three nationwide datasets and developed a two-step prediction model utilizing a machine learning algorithm. Ninety-eight clinical characteristics of patients identified at the pre-hospital stage and 13 hospital components were used as input data for the model. The primary endpoint of the model was the prediction of transfer to an inappropriate hospital.

Results

A total of 94,256 transferred patients in the public pre-hospital care system matched the National Emergency Department Information System data of patients with a pre-hospital cardiovascular registry created in South Korea between July 2017 and December 2018. Of these, 1,770 (6.26%) patients failed to be transferred to a capable hospital. The area under the receiver operating characteristic curve of the final predictive model was 0.813 (0.800–0.825), and the area under the receiver precision-recall curve was 0.286 (0.265–0.308).

Conclusions

Our prediction model used machine learning to show favorable performance in transferring patients with suspected cardiovascular disease to a capable hospital. For our results to lead to changes in the pre-hospital care system, a digital platform for sharing real-time information should be developed.
Appendix
Available only for authorised users
Literature
2.
go back to reference Murray CJ, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, Ezzati M, Shibuya K, Salomon JA, Abdalla S, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the global burden of Disease Study 2010. Lancet. 2012;380(9859):2197–223.CrossRefPubMed Murray CJ, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C, Ezzati M, Shibuya K, Salomon JA, Abdalla S, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the global burden of Disease Study 2010. Lancet. 2012;380(9859):2197–223.CrossRefPubMed
3.
go back to reference Bradley EH, Herrin J, Wang Y, Barton BA, Webster TR, Mattera JA, et al. Strategies for reducing the door-to-balloon time in acute myocardial infarction. N Engl J Med. 2006;355(22):2308–20.CrossRefPubMed Bradley EH, Herrin J, Wang Y, Barton BA, Webster TR, Mattera JA, et al. Strategies for reducing the door-to-balloon time in acute myocardial infarction. N Engl J Med. 2006;355(22):2308–20.CrossRefPubMed
4.
go back to reference Diercks DB, Kontos MC, Chen AY, Pollack CV Jr, Wiviott SD, Rumsfeld JS, et al. Utilization and impact of pre-hospital electrocardiograms for patients with acute ST-segment elevation myocardial infarction: data from the NCDR (National Cardiovascular Data Registry) ACTION (Acute Coronary treatment and intervention outcomes Network) Registry. J Am Coll Cardiol. 2009;53(2):161–6.CrossRefPubMed Diercks DB, Kontos MC, Chen AY, Pollack CV Jr, Wiviott SD, Rumsfeld JS, et al. Utilization and impact of pre-hospital electrocardiograms for patients with acute ST-segment elevation myocardial infarction: data from the NCDR (National Cardiovascular Data Registry) ACTION (Acute Coronary treatment and intervention outcomes Network) Registry. J Am Coll Cardiol. 2009;53(2):161–6.CrossRefPubMed
5.
go back to reference Jollis JG, Roettig ML, Aluko AO, Anstrom KJ, Applegate RJ, Babb JD, et al. Implementation of a statewide system for coronary reperfusion for ST-segment elevation myocardial infarction. JAMA. 2007;298(20):2371–80.CrossRefPubMed Jollis JG, Roettig ML, Aluko AO, Anstrom KJ, Applegate RJ, Babb JD, et al. Implementation of a statewide system for coronary reperfusion for ST-segment elevation myocardial infarction. JAMA. 2007;298(20):2371–80.CrossRefPubMed
6.
go back to reference Kraft PL, Newman S, Hanson D, Anderson W, Bastani A. Emergency physician discretion to activate the cardiac catheterization team decreases door-to-balloon time for acute ST-elevation myocardial infarction. Ann Emerg Med. 2007;50(5):520–6.CrossRefPubMed Kraft PL, Newman S, Hanson D, Anderson W, Bastani A. Emergency physician discretion to activate the cardiac catheterization team decreases door-to-balloon time for acute ST-elevation myocardial infarction. Ann Emerg Med. 2007;50(5):520–6.CrossRefPubMed
7.
go back to reference Kurz MC, Babcock C, Sinha S, Tupesis JP, Allegretti J. The impact of emergency physician-initiated primary percutaneous coronary intervention on mean door-to-balloon time in patients with ST-segment-elevation myocardial infarction. Ann Emerg Med. 2007;50(5):527–34.CrossRefPubMed Kurz MC, Babcock C, Sinha S, Tupesis JP, Allegretti J. The impact of emergency physician-initiated primary percutaneous coronary intervention on mean door-to-balloon time in patients with ST-segment-elevation myocardial infarction. Ann Emerg Med. 2007;50(5):527–34.CrossRefPubMed
8.
go back to reference Park YH, Kang GH, Song BG, Chun WJ, Lee JH, Hwang SY, et al. Factors related to prehospital time delay in acute ST-segment elevation myocardial infarction. J Korean Med Sci. 2012;27(8):864–9.CrossRefPubMedPubMedCentral Park YH, Kang GH, Song BG, Chun WJ, Lee JH, Hwang SY, et al. Factors related to prehospital time delay in acute ST-segment elevation myocardial infarction. J Korean Med Sci. 2012;27(8):864–9.CrossRefPubMedPubMedCentral
9.
go back to reference Frisch A, Heidle KJ, Frisch SO, Ata A, Kramer B, Colleran C, et al. Factors associated with advanced cardiac care in prehospital chest pain patients. Am J Emerg Med. 2018;36(7):1182–7.CrossRefPubMed Frisch A, Heidle KJ, Frisch SO, Ata A, Kramer B, Colleran C, et al. Factors associated with advanced cardiac care in prehospital chest pain patients. Am J Emerg Med. 2018;36(7):1182–7.CrossRefPubMed
10.
go back to reference Stopyra JP, Harper WS, Higgins TJ, Prokesova JV, Winslow JE, Nelson RD, et al. Prehospital modified HEART score predictive of 30-Day adverse cardiac events. Prehosp Disaster Med. 2018;33(1):58–62.CrossRefPubMed Stopyra JP, Harper WS, Higgins TJ, Prokesova JV, Winslow JE, Nelson RD, et al. Prehospital modified HEART score predictive of 30-Day adverse cardiac events. Prehosp Disaster Med. 2018;33(1):58–62.CrossRefPubMed
11.
go back to reference Wibring K, Herlitz J, Christensson L, Lingman M, Bång A. Prehospital factors associated with an acute life-threatening condition in non-traumatic chest pain patients - a systematic review. Int J Cardiol. 2016;219:373–9.CrossRefPubMed Wibring K, Herlitz J, Christensson L, Lingman M, Bång A. Prehospital factors associated with an acute life-threatening condition in non-traumatic chest pain patients - a systematic review. Int J Cardiol. 2016;219:373–9.CrossRefPubMed
12.
go back to reference Rawshani N, Rawshani A, Gelang C, Herlitz J, Bång A, Andersson JO, et al. Association between use of pre-hospital ECG and 30-day mortality: a large cohort study of patients experiencing chest pain. Int J Cardiol. 2017;248:77–81.CrossRefPubMed Rawshani N, Rawshani A, Gelang C, Herlitz J, Bång A, Andersson JO, et al. Association between use of pre-hospital ECG and 30-day mortality: a large cohort study of patients experiencing chest pain. Int J Cardiol. 2017;248:77–81.CrossRefPubMed
13.
go back to reference Karam N, Bataille S, Marijon E, Giovannetti O, Tafflet M, Savary D, et al. Identifying patients at risk for prehospital sudden cardiac arrest at the early phase of myocardial infarction: the e-MUST study (evaluation en Médecine d’Urgence des Stratégies Thérapeutiques des infarctus du myocarde). Circulation. 2016;134(25):2074–83.CrossRefPubMed Karam N, Bataille S, Marijon E, Giovannetti O, Tafflet M, Savary D, et al. Identifying patients at risk for prehospital sudden cardiac arrest at the early phase of myocardial infarction: the e-MUST study (evaluation en Médecine d’Urgence des Stratégies Thérapeutiques des infarctus du myocarde). Circulation. 2016;134(25):2074–83.CrossRefPubMed
14.
go back to reference Cho KJ, Kwon O, Kwon JM, Lee Y, Park H, Jeon KH, et al. Detecting patient deterioration using artificial intelligence in a rapid response system. Crit Care Med. 2020;48(4):e285–e9.CrossRefPubMed Cho KJ, Kwon O, Kwon JM, Lee Y, Park H, Jeon KH, et al. Detecting patient deterioration using artificial intelligence in a rapid response system. Crit Care Med. 2020;48(4):e285–e9.CrossRefPubMed
15.
go back to reference Kwon JM, Jeon KH, Kim HM, Kim MJ, Lim S, Kim KH, et al. Deep-learning-based out-of-hospital cardiac arrest prognostic system to predict clinical outcomes. Resuscitation. 2019;139:84–91.CrossRefPubMed Kwon JM, Jeon KH, Kim HM, Kim MJ, Lim S, Kim KH, et al. Deep-learning-based out-of-hospital cardiac arrest prognostic system to predict clinical outcomes. Resuscitation. 2019;139:84–91.CrossRefPubMed
16.
go back to reference Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc. 2018;7(13). Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc. 2018;7(13).
17.
go back to reference Ro YS, Shin SD, Lee YJ, Lee SC, Song KJ, Ryoo HW, et al. Effect of dispatcher-assisted cardiopulmonary resuscitation program and location of out-of-hospital cardiac arrest on survival and neurologic outcome. Ann Emerg Med. 2017;69(1):52–61.CrossRefPubMed Ro YS, Shin SD, Lee YJ, Lee SC, Song KJ, Ryoo HW, et al. Effect of dispatcher-assisted cardiopulmonary resuscitation program and location of out-of-hospital cardiac arrest on survival and neurologic outcome. Ann Emerg Med. 2017;69(1):52–61.CrossRefPubMed
18.
go back to reference Kim EN, Kim MJ, You JS, Shin HJ, Park IC, Chung SP, et al. Effects of an emergency transfer coordination center on secondary overtriage in an emergency department. Am J Emerg Med. 2019;37(3):395–400.CrossRefPubMed Kim EN, Kim MJ, You JS, Shin HJ, Park IC, Chung SP, et al. Effects of an emergency transfer coordination center on secondary overtriage in an emergency department. Am J Emerg Med. 2019;37(3):395–400.CrossRefPubMed
19.
go back to reference Kim JH, Han SG, Cho A, Shin HJ, Baek SE. Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study. BMC Med Inform Decis Mak. 2021;21(1):311.CrossRefPubMedPubMedCentral Kim JH, Han SG, Cho A, Shin HJ, Baek SE. Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study. BMC Med Inform Decis Mak. 2021;21(1):311.CrossRefPubMedPubMedCentral
20.
go back to reference Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2(1):56–67.CrossRefPubMedPubMedCentral Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020;2(1):56–67.CrossRefPubMedPubMedCentral
21.
go back to reference Żurowska-Wolak M, Piekos P, Jąkała J, Mikos M. The effects of prehospital system delays on the treatment efficacy of STEMI patients. Scand J Trauma Resusc Emerg Med. 2019;27(1):39.CrossRefPubMedPubMedCentral Żurowska-Wolak M, Piekos P, Jąkała J, Mikos M. The effects of prehospital system delays on the treatment efficacy of STEMI patients. Scand J Trauma Resusc Emerg Med. 2019;27(1):39.CrossRefPubMedPubMedCentral
22.
go back to reference Foo CY, Bonsu KO, Nallamothu BK, Reid CM, Dhippayom T, Reidpath DD, et al. Coronary intervention door-to-balloon time and outcomes in ST-elevation myocardial infarction: a meta-analysis. Heart. 2018;104(16):1362–9.CrossRefPubMed Foo CY, Bonsu KO, Nallamothu BK, Reid CM, Dhippayom T, Reidpath DD, et al. Coronary intervention door-to-balloon time and outcomes in ST-elevation myocardial infarction: a meta-analysis. Heart. 2018;104(16):1362–9.CrossRefPubMed
23.
go back to reference Grzybowski M, Zalenski RJ, Ross MA, Bock B. A prediction model for prehospital triage of patients with suspected cardiac ischemia. J Electrocardiol. 2000;33:253–8.CrossRefPubMed Grzybowski M, Zalenski RJ, Ross MA, Bock B. A prediction model for prehospital triage of patients with suspected cardiac ischemia. J Electrocardiol. 2000;33:253–8.CrossRefPubMed
24.
go back to reference Kim JH, Kim MJ, You JS, Song MK, Cho SI. Do emergency physicians improve the appropriateness of emergency transfer in rural areas? J Emerg Med. 2018;54(3):287–94.CrossRefPubMed Kim JH, Kim MJ, You JS, Song MK, Cho SI. Do emergency physicians improve the appropriateness of emergency transfer in rural areas? J Emerg Med. 2018;54(3):287–94.CrossRefPubMed
25.
go back to reference Holland CM, Lovasik BP, Howard BM, McClure EW, Samuels OB, Barrow DL. Interhospital transfer of neurosurgical patients: implications of timing on hospital course and clinical outcomes. Neurosurgery. 2017;81(3):450–7.CrossRefPubMed Holland CM, Lovasik BP, Howard BM, McClure EW, Samuels OB, Barrow DL. Interhospital transfer of neurosurgical patients: implications of timing on hospital course and clinical outcomes. Neurosurgery. 2017;81(3):450–7.CrossRefPubMed
26.
go back to reference Javat D, Heal C, Banks J, Buchholz S, Zhang Z. Regional to tertiary inter-hospital transfer versus in-house percutaneous coronary intervention in acute coronary syndrome. PLoS ONE. 2018;13(6):e0198272.CrossRefPubMedPubMedCentral Javat D, Heal C, Banks J, Buchholz S, Zhang Z. Regional to tertiary inter-hospital transfer versus in-house percutaneous coronary intervention in acute coronary syndrome. PLoS ONE. 2018;13(6):e0198272.CrossRefPubMedPubMedCentral
27.
go back to reference Sorensen MJ, von Recklinghausen FM, Fulton G, Burchard KW. Secondary overtriage: the burden of unnecessary interfacility transfers in a rural trauma system. JAMA Surg. 2013;148(8):763–8.CrossRefPubMed Sorensen MJ, von Recklinghausen FM, Fulton G, Burchard KW. Secondary overtriage: the burden of unnecessary interfacility transfers in a rural trauma system. JAMA Surg. 2013;148(8):763–8.CrossRefPubMed
28.
go back to reference Bible JE, Kadakia RJ, Kay HF, Zhang CE, Casimir GE, Devin CJ. How often are interfacility transfers of spine injury patients truly necessary? Spine J. 2014;14(12):2877–84.CrossRefPubMed Bible JE, Kadakia RJ, Kay HF, Zhang CE, Casimir GE, Devin CJ. How often are interfacility transfers of spine injury patients truly necessary? Spine J. 2014;14(12):2877–84.CrossRefPubMed
29.
go back to reference Lee SJ, Choi A, Ryoo HW, Pak YS, Kim HC, Kim JH. Changes in clinical characteristics among febrile patients visiting the emergency department before and after the COVID-19 outbreak. Yonsei Med J. 2021;62(12):1136–44.CrossRefPubMedPubMedCentral Lee SJ, Choi A, Ryoo HW, Pak YS, Kim HC, Kim JH. Changes in clinical characteristics among febrile patients visiting the emergency department before and after the COVID-19 outbreak. Yonsei Med J. 2021;62(12):1136–44.CrossRefPubMedPubMedCentral
30.
go back to reference Al-Zaiti S, Besomi L, Bouzid Z, Faramand Z, Frisch S, Martin-Gill C, et al. Machine learning-based prediction of acute coronary syndrome using only the pre-hospital 12-lead electrocardiogram. Nat Commun. 2020;11(1):3966.CrossRefPubMedPubMedCentral Al-Zaiti S, Besomi L, Bouzid Z, Faramand Z, Frisch S, Martin-Gill C, et al. Machine learning-based prediction of acute coronary syndrome using only the pre-hospital 12-lead electrocardiogram. Nat Commun. 2020;11(1):3966.CrossRefPubMedPubMedCentral
31.
go back to reference Kim H, Kim S-W, Park E, Kim JH, Chang H. The role of fifth-generation mobile technology in prehospital emergency care: an opportunity to support paramedics. Health Policy and Technology. 2020;9(1):109–14.CrossRef Kim H, Kim S-W, Park E, Kim JH, Chang H. The role of fifth-generation mobile technology in prehospital emergency care: an opportunity to support paramedics. Health Policy and Technology. 2020;9(1):109–14.CrossRef
32.
go back to reference Schwartz JM, George M, Rossetti SC, Dykes PC, Minshall SR, Lucas E, et al. Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study. JMIR Hum Factors. 2022;9(2):e33960.CrossRefPubMedPubMedCentral Schwartz JM, George M, Rossetti SC, Dykes PC, Minshall SR, Lucas E, et al. Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study. JMIR Hum Factors. 2022;9(2):e33960.CrossRefPubMedPubMedCentral
33.
go back to reference Muralitharan S, Nelson W, Di S, McGillion M, Devereaux PJ, Barr NG, et al. Machine learning-based early warning systems for clinical deterioration: systematic scoping review. J Med Internet Res. 2021;23(2):e25187.CrossRefPubMedPubMedCentral Muralitharan S, Nelson W, Di S, McGillion M, Devereaux PJ, Barr NG, et al. Machine learning-based early warning systems for clinical deterioration: systematic scoping review. J Med Internet Res. 2021;23(2):e25187.CrossRefPubMedPubMedCentral
34.
go back to reference Prokhorenkova L, Gusev G, Vorobev A, Dorogush AV, Gulin A. CatBoost: unbiased boosting with categorical features. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal, Canada: Curran Associates Inc.; 2018. p. 6639–49. Prokhorenkova L, Gusev G, Vorobev A, Dorogush AV, Gulin A. CatBoost: unbiased boosting with categorical features. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal, Canada: Curran Associates Inc.; 2018. p. 6639–49.
35.
go back to reference Shwartz-Ziv R, Armon A. Tabular data: deep learning is not all you need. Inf Fusion. 2022;81(C):84–90.CrossRef Shwartz-Ziv R, Armon A. Tabular data: deep learning is not all you need. Inf Fusion. 2022;81(C):84–90.CrossRef
Metadata
Title
Prediction of inappropriate pre-hospital transfer of patients with suspected cardiovascular emergency diseases using machine learning: a retrospective observational study
Authors
Ji Hoon Kim
Bomgyeol Kim
Min Joung Kim
Heejung Hyun
Hyeon Chang Kim
Hyuk-Jae Chang
Publication date
01-12-2023
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-023-02149-9

Other articles of this Issue 1/2023

BMC Medical Informatics and Decision Making 1/2023 Go to the issue