Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2014

Open Access 01-12-2014 | Research article

Prediction of adverse cardiac events in emergency department patients with chest pain using machine learning for variable selection

Authors: Nan Liu, Zhi Xiong Koh, Junyang Goh, Zhiping Lin, Benjamin Haaland, Boon Ping Ting, Marcus Eng Hock Ong

Published in: BMC Medical Informatics and Decision Making | Issue 1/2014

Login to get access

Abstract

Background

The key aim of triage in chest pain patients is to identify those with high risk of adverse cardiac events as they require intensive monitoring and early intervention. In this study, we aim to discover the most relevant variables for risk prediction of major adverse cardiac events (MACE) using clinical signs and heart rate variability.

Methods

A total of 702 chest pain patients at the Emergency Department (ED) of a tertiary hospital in Singapore were included in this study. The recruited patients were at least 30 years of age and who presented to the ED with a primary complaint of non-traumatic chest pain. The primary outcome was a composite of MACE such as death and cardiac arrest within 72 h of arrival at the ED. For each patient, eight clinical signs such as blood pressure and temperature were measured, and a 5-min ECG was recorded to derive heart rate variability parameters. A random forest-based novel method was developed to select the most relevant variables. A geometric distance-based machine learning scoring system was then implemented to derive a risk score from 0 to 100.

Results

Out of 702 patients, 29 (4.1%) met the primary outcome. We selected the 3 most relevant variables for predicting MACE, which were systolic blood pressure, the mean RR interval and the mean instantaneous heart rate. The scoring system with these 3 variables produced an area under the curve (AUC) of 0.812, and a cutoff score of 43 gave a sensitivity of 82.8% and specificity of 63.4%, while the scoring system with all the 23 variables had an AUC of 0.736, and a cutoff score of 49 gave a sensitivity of 72.4% and specificity of 63.0%. Conventional thrombolysis in myocardial infarction score and the modified early warning score achieved AUC values of 0.637 and 0.622, respectively.

Conclusions

It is observed that a few predictors outperformed the whole set of variables in predicting MACE within 72 h. We conclude that more predictors do not necessarily guarantee better prediction results. Furthermore, machine learning-based variable selection seems promising in discovering a few relevant and significant measures as predictors.
Appendix
Available only for authorised users
Literature
1.
go back to reference McCaig LF, Burt CW: National Hospital Ambulatory Medical Care Survey: 2001 emergency department summary. Adv Data. 2003, 335: 1-29.PubMed McCaig LF, Burt CW: National Hospital Ambulatory Medical Care Survey: 2001 emergency department summary. Adv Data. 2003, 335: 1-29.PubMed
2.
go back to reference Anderson JL, Adams CD, Antman EM, Bridges CR, Califf RM, Casey DE, Chavey WE, Fesmire FM, Hochman JS, Levin TN, Lincoff AM, Peterson ED, Theroux P, Wenger NK, Wright RS, Smith SC, Jacobs AK, Halperin JL, Hunt SA, Krumholz HM, Kushner FG, Lytle BW, Nishimura R, Ornato JP, Page RL, Riegel B: ACC/AHA 2007 guidelines for the management of patients with unstable angina/non-ST-Elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines for the Management of Patients With Unstable Angina/Non-ST-Elevation Myocardial Infarction) developed in collaboration with the American College of Emergency Physicians, the Society for Cardiovascular Angiography and Interventions, and the Society of Thoracic Surgeons endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation and the Society for Academic Emergency Medicine. J Am Coll Cardiol. 2007, 50 (7): e1-e157.CrossRefPubMed Anderson JL, Adams CD, Antman EM, Bridges CR, Califf RM, Casey DE, Chavey WE, Fesmire FM, Hochman JS, Levin TN, Lincoff AM, Peterson ED, Theroux P, Wenger NK, Wright RS, Smith SC, Jacobs AK, Halperin JL, Hunt SA, Krumholz HM, Kushner FG, Lytle BW, Nishimura R, Ornato JP, Page RL, Riegel B: ACC/AHA 2007 guidelines for the management of patients with unstable angina/non-ST-Elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines for the Management of Patients With Unstable Angina/Non-ST-Elevation Myocardial Infarction) developed in collaboration with the American College of Emergency Physicians, the Society for Cardiovascular Angiography and Interventions, and the Society of Thoracic Surgeons endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation and the Society for Academic Emergency Medicine. J Am Coll Cardiol. 2007, 50 (7): e1-e157.CrossRefPubMed
3.
go back to reference Huikuri HV, Castellanos A, Myerburg RJ: Sudden death due to cardiac arrhythmias. N Engl J Med. 2001, 345 (20): 1473-1482.CrossRefPubMed Huikuri HV, Castellanos A, Myerburg RJ: Sudden death due to cardiac arrhythmias. N Engl J Med. 2001, 345 (20): 1473-1482.CrossRefPubMed
4.
go back to reference Goldman L, Cook EF, Johnson PA, Brand DA, Rouan GW, Lee TH: Prediction of the need for intensive care in patients who come to the emergency departments with acute chest pain. N Engl J Med. 1996, 334 (23): 1498-1504.CrossRefPubMed Goldman L, Cook EF, Johnson PA, Brand DA, Rouan GW, Lee TH: Prediction of the need for intensive care in patients who come to the emergency departments with acute chest pain. N Engl J Med. 1996, 334 (23): 1498-1504.CrossRefPubMed
5.
go back to reference Antman E, Cohen M, Bernink P, McCabe C, Horacek T, Papuchis G, Mautner B, Corbalan R, Radley D, Braunwald E: The TIMI risk score for unstable angina/non-ST elevation MI - A method for prognostication and therapeutic decision making. JAMA. 2000, 284 (7): 835-842.CrossRefPubMed Antman E, Cohen M, Bernink P, McCabe C, Horacek T, Papuchis G, Mautner B, Corbalan R, Radley D, Braunwald E: The TIMI risk score for unstable angina/non-ST elevation MI - A method for prognostication and therapeutic decision making. JAMA. 2000, 284 (7): 835-842.CrossRefPubMed
6.
go back to reference Eagle KA, Lim MJ, Dabbous OH, Pieper KS, Goldberg RJ, Van de Werf F, Goodman SG, Granger CB, Steg PG, Gore JM, Budaj A, Avezum A, Flather MD, Fox KA: A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry. JAMA. 2004, 291 (22): 2727-2733.CrossRefPubMed Eagle KA, Lim MJ, Dabbous OH, Pieper KS, Goldberg RJ, Van de Werf F, Goodman SG, Granger CB, Steg PG, Gore JM, Budaj A, Avezum A, Flather MD, Fox KA: A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry. JAMA. 2004, 291 (22): 2727-2733.CrossRefPubMed
7.
go back to reference Lyon R, Morris AC, Caesar D, Gray S, Gray A: Chest pain presenting to the Emergency Department–to stratify risk with GRACE or TIMI?. Resuscitation. 2007, 74 (1): 90-93.CrossRefPubMed Lyon R, Morris AC, Caesar D, Gray S, Gray A: Chest pain presenting to the Emergency Department–to stratify risk with GRACE or TIMI?. Resuscitation. 2007, 74 (1): 90-93.CrossRefPubMed
8.
go back to reference Subbe CP, Kruger M, Rutherford P, Gemmel L: Validation of a modified early warning score in medical admissions. QJM. 2001, 94 (10): 521-526.CrossRefPubMed Subbe CP, Kruger M, Rutherford P, Gemmel L: Validation of a modified early warning score in medical admissions. QJM. 2001, 94 (10): 521-526.CrossRefPubMed
9.
go back to reference Goldhill DR, McNarry AF, Mandersloot G, McGinley A: A physiologically-based early warning score for ward patients: the association between score and outcome. Anaesthesia. 2005, 60 (6): 547-553.CrossRefPubMed Goldhill DR, McNarry AF, Mandersloot G, McGinley A: A physiologically-based early warning score for ward patients: the association between score and outcome. Anaesthesia. 2005, 60 (6): 547-553.CrossRefPubMed
10.
go back to reference Ong MEH, Ng CHL, Goh K, Liu N, Koh ZX, Shahidah N, Zhang T, Fook-Chong S, Lin Z: Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a machine learning score incorporating heart rate variability compared with the modified early warning score. Crit Care. 2012, 16 (3): R108-CrossRefPubMedPubMedCentral Ong MEH, Ng CHL, Goh K, Liu N, Koh ZX, Shahidah N, Zhang T, Fook-Chong S, Lin Z: Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a machine learning score incorporating heart rate variability compared with the modified early warning score. Crit Care. 2012, 16 (3): R108-CrossRefPubMedPubMedCentral
11.
go back to reference Manini AF, Dannemann N, Brown DF, Butter J, Bamberg F, Nagurney JT, Nichols JH, Hoffmann U, Rule-Out Myocardial Infarction U: Limitations of risk score models in patients with acute chest pain. Am J Emerg Med. 2009, 27 (1): 43-48.CrossRefPubMedPubMedCentral Manini AF, Dannemann N, Brown DF, Butter J, Bamberg F, Nagurney JT, Nichols JH, Hoffmann U, Rule-Out Myocardial Infarction U: Limitations of risk score models in patients with acute chest pain. Am J Emerg Med. 2009, 27 (1): 43-48.CrossRefPubMedPubMedCentral
12.
go back to reference Hollander JE, Robey JL, Chase MR, Brown AM, Zogby KE, Shofer FS: Relationship between a clear-cut alternative noncardiac diagnosis and 30-day outcome in emergency department patients with chest pain. Acad Emerg Med. 2007, 14 (3): 210-215.CrossRefPubMed Hollander JE, Robey JL, Chase MR, Brown AM, Zogby KE, Shofer FS: Relationship between a clear-cut alternative noncardiac diagnosis and 30-day outcome in emergency department patients with chest pain. Acad Emerg Med. 2007, 14 (3): 210-215.CrossRefPubMed
13.
go back to reference Ong MEH, Goh K, Fook-Chong S, Haaland B, Wai KL, Koh ZX, Shahidah N, Lin Z: Heart rate variability risk score for prediction of acute cardiac complications in ED patients with chest pain. Am J Emerg Med. 2013, 31 (8): 1201-1207.CrossRefPubMed Ong MEH, Goh K, Fook-Chong S, Haaland B, Wai KL, Koh ZX, Shahidah N, Lin Z: Heart rate variability risk score for prediction of acute cardiac complications in ED patients with chest pain. Am J Emerg Med. 2013, 31 (8): 1201-1207.CrossRefPubMed
14.
go back to reference Sanchis J, Bodi V, Nunez J, Bosch X, Lorna-Sorio P, Mainar L, Santas E, Minana G, Robles R, Llacer A: Limitations of clinical history for evaluation of patients with acute chest pain, non-diagnostic electrocardiogram, and normal troponin. Am J Cardiol. 2008, 101 (5): 613-617.CrossRefPubMed Sanchis J, Bodi V, Nunez J, Bosch X, Lorna-Sorio P, Mainar L, Santas E, Minana G, Robles R, Llacer A: Limitations of clinical history for evaluation of patients with acute chest pain, non-diagnostic electrocardiogram, and normal troponin. Am J Cardiol. 2008, 101 (5): 613-617.CrossRefPubMed
15.
go back to reference Hargarten KM, Aprahamian C, Stueven H, Olson DW, Aufderheide TP, Mateer JR: Limitations of prehospital predictors of acute myocardial infarction and unstable angina. Ann Emerg Med. 1987, 16 (12): 1325-1329.CrossRefPubMed Hargarten KM, Aprahamian C, Stueven H, Olson DW, Aufderheide TP, Mateer JR: Limitations of prehospital predictors of acute myocardial infarction and unstable angina. Ann Emerg Med. 1987, 16 (12): 1325-1329.CrossRefPubMed
16.
go back to reference Pearce CB, Gunn SR, Ahmed A, Johnson CD: Machine learning can improve prediction of severity in acute pancreatitis using admission values of APACHE II score and C-reactive protein. Pancreatology. 2006, 6 (1–2): 123-131.CrossRefPubMed Pearce CB, Gunn SR, Ahmed A, Johnson CD: Machine learning can improve prediction of severity in acute pancreatitis using admission values of APACHE II score and C-reactive protein. Pancreatology. 2006, 6 (1–2): 123-131.CrossRefPubMed
17.
go back to reference Liu N, Lin Z, Cao J, Koh ZX, Zhang T, Huang G-B, Ser W, Ong MEH: An intelligent scoring system and its application to cardiac arrest prediction. IEEE Trans Inf Technol Biomed. 2012, 16 (6): 1324-1331.CrossRefPubMed Liu N, Lin Z, Cao J, Koh ZX, Zhang T, Huang G-B, Ser W, Ong MEH: An intelligent scoring system and its application to cardiac arrest prediction. IEEE Trans Inf Technol Biomed. 2012, 16 (6): 1324-1331.CrossRefPubMed
18.
go back to reference Liu H, Yu L: Toward integrating feature selection algorithms for classification and clustering. IEEE Trans Knowl Data Eng. 2005, 17 (4): 491-502.CrossRef Liu H, Yu L: Toward integrating feature selection algorithms for classification and clustering. IEEE Trans Knowl Data Eng. 2005, 17 (4): 491-502.CrossRef
19.
go back to reference Huynh-Thu VA, Saeys Y, Wehenkel L, Geurts P: Statistical interpretation of machine learning-based feature importance scores for biomarker discovery. Bioinformatics. 2012, 28 (13): 1766-1774.CrossRefPubMed Huynh-Thu VA, Saeys Y, Wehenkel L, Geurts P: Statistical interpretation of machine learning-based feature importance scores for biomarker discovery. Bioinformatics. 2012, 28 (13): 1766-1774.CrossRefPubMed
20.
go back to reference Katz MH: Multivariable analysis: a primer for readers of medical research. Ann Intern Med. 2003, 138 (8): 644-650.CrossRefPubMed Katz MH: Multivariable analysis: a primer for readers of medical research. Ann Intern Med. 2003, 138 (8): 644-650.CrossRefPubMed
21.
go back to reference Fawcett T: An introduction to ROC analysis. Pattern Recognit Lett. 2006, 27 (8): 861-874.CrossRef Fawcett T: An introduction to ROC analysis. Pattern Recognit Lett. 2006, 27 (8): 861-874.CrossRef
22.
go back to reference Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology: Heart rate variability: standards of measurement, physiological interpretation and clinical use. Circulation. 1996, 93 (5): 1043-1065.CrossRef Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology: Heart rate variability: standards of measurement, physiological interpretation and clinical use. Circulation. 1996, 93 (5): 1043-1065.CrossRef
23.
go back to reference Austin PC, Tu JV: Automated variable selection methods for logistic regression produced unstable models for predicting acute myocardial infarction mortality. J Clin Epidemiol. 2004, 57 (11): 1138-1146.CrossRefPubMed Austin PC, Tu JV: Automated variable selection methods for logistic regression produced unstable models for predicting acute myocardial infarction mortality. J Clin Epidemiol. 2004, 57 (11): 1138-1146.CrossRefPubMed
24.
go back to reference Liu N, Lin Z, Koh ZX, Huang G-B, Ser W, Ong MEH: Patient outcome prediction with heart rate variability and vital signs. J Signal Process Syst. 2011, 64: 265-278.CrossRef Liu N, Lin Z, Koh ZX, Huang G-B, Ser W, Ong MEH: Patient outcome prediction with heart rate variability and vital signs. J Signal Process Syst. 2011, 64: 265-278.CrossRef
25.
go back to reference He H, Garcia EA: Learning from imbalanced data. IEEE Trans Knowl Data Eng. 2009, 21 (9): 1263-1284.CrossRef He H, Garcia EA: Learning from imbalanced data. IEEE Trans Knowl Data Eng. 2009, 21 (9): 1263-1284.CrossRef
27.
go back to reference Genuer R, Poggi J-M, Tuleau-Malot C: Variable selection using random forests. Pattern Recogn Lett. 2010, 31 (14): 2225-2236.CrossRef Genuer R, Poggi J-M, Tuleau-Malot C: Variable selection using random forests. Pattern Recogn Lett. 2010, 31 (14): 2225-2236.CrossRef
28.
go back to reference Hapfelmeier A, Ulm K: A new variable selection approach using Random Forests. Computational Statistics & Data Analysis. 2013, 60: 50-69.CrossRef Hapfelmeier A, Ulm K: A new variable selection approach using Random Forests. Computational Statistics & Data Analysis. 2013, 60: 50-69.CrossRef
29.
go back to reference Kuhn M: Building predictive models in R using the caret package. J Stat Softw. 2008, 28 (5): 1-26.CrossRef Kuhn M: Building predictive models in R using the caret package. J Stat Softw. 2008, 28 (5): 1-26.CrossRef
30.
go back to reference Han J, Kamber M: Data Mining: Concepts and Techniques. 2006, San Francisco: Morgan Kaufmann Han J, Kamber M: Data Mining: Concepts and Techniques. 2006, San Francisco: Morgan Kaufmann
31.
go back to reference Burges C: A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov. 1998, 2 (2): 121-167.CrossRef Burges C: A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov. 1998, 2 (2): 121-167.CrossRef
32.
go back to reference Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, Muller M: pROC: an open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics. 2011, 12: 77-CrossRefPubMedPubMedCentral Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, Muller M: pROC: an open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics. 2011, 12: 77-CrossRefPubMedPubMedCentral
33.
go back to reference Hanson CW, Marshall BE: Artificial intelligence applications in the intensive care unit. Crit Care Med. 2001, 29 (2): 427-435.CrossRefPubMed Hanson CW, Marshall BE: Artificial intelligence applications in the intensive care unit. Crit Care Med. 2001, 29 (2): 427-435.CrossRefPubMed
34.
go back to reference Hong WL, Earnest A, Sultana P, Koh ZX, Shahidah N, Ong MEH: How accurate are vital signs in predicting clinical outcomes in critically ill emergency department patients. Eur J Emerg Med. 2013, 20 (1): 27-32.CrossRefPubMed Hong WL, Earnest A, Sultana P, Koh ZX, Shahidah N, Ong MEH: How accurate are vital signs in predicting clinical outcomes in critically ill emergency department patients. Eur J Emerg Med. 2013, 20 (1): 27-32.CrossRefPubMed
36.
go back to reference Polanczyk CA, Lee TH, Cook EF, Walls R, Wybenga D, Printy-Klein G, Ludwig L, Guldbrandsen G, Johnson PA: Cardiac troponin I as a predictor of major cardiac events in emergency department patients with acute chest pain. J Am Coll Cardiol. 1998, 32 (1): 8-14.CrossRefPubMed Polanczyk CA, Lee TH, Cook EF, Walls R, Wybenga D, Printy-Klein G, Ludwig L, Guldbrandsen G, Johnson PA: Cardiac troponin I as a predictor of major cardiac events in emergency department patients with acute chest pain. J Am Coll Cardiol. 1998, 32 (1): 8-14.CrossRefPubMed
Metadata
Title
Prediction of adverse cardiac events in emergency department patients with chest pain using machine learning for variable selection
Authors
Nan Liu
Zhi Xiong Koh
Junyang Goh
Zhiping Lin
Benjamin Haaland
Boon Ping Ting
Marcus Eng Hock Ong
Publication date
01-12-2014
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2014
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/1472-6947-14-75

Other articles of this Issue 1/2014

BMC Medical Informatics and Decision Making 1/2014 Go to the issue