Skip to main content
Top
Published in: Critical Care 1/2021

01-12-2021 | Care | Research

Predicting neurological outcome after out-of-hospital cardiac arrest with cumulative information; development and internal validation of an artificial neural network algorithm

Authors: Peder Andersson, Jesper Johnsson, Ola Björnsson, Tobias Cronberg, Christian Hassager, Henrik Zetterberg, Pascal Stammet, Johan Undén, Jesper Kjaergaard, Hans Friberg, Kaj Blennow, Gisela Lilja, Matt P. Wise, Josef Dankiewicz, Niklas Nielsen, Attila Frigyesi

Published in: Critical Care | Issue 1/2021

Login to get access

Abstract

Background

Prognostication of neurological outcome in patients who remain comatose after cardiac arrest resuscitation is complex. Clinical variables, as well as biomarkers of brain injury, cardiac injury, and systemic inflammation, all yield some prognostic value. We hypothesised that cumulative information obtained during the first three days of intensive care could produce a reliable model for predicting neurological outcome following out-of-hospital cardiac arrest (OHCA) using artificial neural network (ANN) with and without biomarkers.

Methods

We performed a post hoc analysis of 932 patients from the Target Temperature Management trial. We focused on comatose patients at 24, 48, and 72 h post-cardiac arrest and excluded patients who were awake or deceased at these time points. 80% of the patients were allocated for model development (training set) and 20% for internal validation (test set). To investigate the prognostic potential of different levels of biomarkers (clinically available and research-grade), patients’ background information, and intensive care observation and treatment, we created three models for each time point: (1) clinical variables, (2) adding clinically accessible biomarkers, e.g., neuron-specific enolase (NSE) and (3) adding research-grade biomarkers, e.g., neurofilament light (NFL). Patient outcome was the dichotomised Cerebral Performance Category (CPC) at six months; a good outcome was defined as CPC 1–2 whilst a poor outcome was defined as CPC 3–5. The area under the receiver operating characteristic curve (AUROC) was calculated for all test sets.

Results

AUROC remained below 90% when using only clinical variables throughout the first three days in the ICU. Adding clinically accessible biomarkers such as NSE, AUROC increased from 82 to 94% (p < 0.01). The prognostic accuracy remained excellent from day 1 to day 3 with an AUROC at approximately 95% when adding research-grade biomarkers. The models which included NSE after 72 h and NFL on any of the three days had a low risk of false-positive predictions while retaining a low number of false-negative predictions.

Conclusions

In this exploratory study, ANNs provided good to excellent prognostic accuracy in predicting neurological outcome in comatose patients post OHCA. The models which included NSE after 72 h and NFL on all days showed promising prognostic performance.
Appendix
Available only for authorised users
Literature
1.
go back to reference Cronberg T, Greer DM, Lilja G, Moulaert V, Swindell P, Rossetti AO. Brain injury after cardiac arrest: from prognostication of comatose patients to rehabilitation. Lancet Neurol. 2020;19(7):611–22.CrossRef Cronberg T, Greer DM, Lilja G, Moulaert V, Swindell P, Rossetti AO. Brain injury after cardiac arrest: from prognostication of comatose patients to rehabilitation. Lancet Neurol. 2020;19(7):611–22.CrossRef
2.
go back to reference Nolan JP, Soar J, Cariou A, Cronberg T, Moulaert VR, Deakin CD, et al. European resuscitation council and European society of intensive care medicine 2015 guidelines for post-resuscitation care. Intensive Care Med. 2015;41(12):2039–56.CrossRef Nolan JP, Soar J, Cariou A, Cronberg T, Moulaert VR, Deakin CD, et al. European resuscitation council and European society of intensive care medicine 2015 guidelines for post-resuscitation care. Intensive Care Med. 2015;41(12):2039–56.CrossRef
3.
go back to reference Nolan JP, Cariou A. Post-resuscitation care: ERC-ESICM guidelines 2015. Intensive Care Med. 2015;41(12):2204–6.CrossRef Nolan JP, Cariou A. Post-resuscitation care: ERC-ESICM guidelines 2015. Intensive Care Med. 2015;41(12):2204–6.CrossRef
4.
go back to reference Sandroni C, D’Arrigo S, Nolan JP. Prognostication after cardiac arrest. Crit Care. 2018;22(1):150.CrossRef Sandroni C, D’Arrigo S, Nolan JP. Prognostication after cardiac arrest. Crit Care. 2018;22(1):150.CrossRef
5.
go back to reference Moseby-Knappe M, Mattsson N, Nielsen N, Zetterberg H, Blennow K, Dankiewicz J, et al. Serum neurofilament light chain for prognosis of outcome after cardiac arrest. JAMA Neurol. 2019;76(1):64–71.CrossRef Moseby-Knappe M, Mattsson N, Nielsen N, Zetterberg H, Blennow K, Dankiewicz J, et al. Serum neurofilament light chain for prognosis of outcome after cardiac arrest. JAMA Neurol. 2019;76(1):64–71.CrossRef
6.
go back to reference Wihersaari L, Ashton NJ, Reinikainen M, Jakkula P, Pettilä V, Hästbacka J, et al. Neurofilament light as an outcome predictor after cardiac arrest: a post hoc analysis of the COMACARE trial. Intensive Care Medicine. 2020. Wihersaari L, Ashton NJ, Reinikainen M, Jakkula P, Pettilä V, Hästbacka J, et al. Neurofilament light as an outcome predictor after cardiac arrest: a post hoc analysis of the COMACARE trial. Intensive Care Medicine. 2020.
7.
go back to reference Kim MJ, Kim T, Suh GJ, Kwon WY, Kim KS, Jung YS, et al. Association between the simultaneous decrease in the levels of soluble vascular cell adhesion molecule-1 and S100 protein and good neurological outcomes in cardiac arrest survivors. Clin Exp Emerg Med. 2018;5(4):211–8.CrossRef Kim MJ, Kim T, Suh GJ, Kwon WY, Kim KS, Jung YS, et al. Association between the simultaneous decrease in the levels of soluble vascular cell adhesion molecule-1 and S100 protein and good neurological outcomes in cardiac arrest survivors. Clin Exp Emerg Med. 2018;5(4):211–8.CrossRef
8.
go back to reference Mattsson N, Zetterberg H, Nielsen N, Blennow K, Dankiewicz J, Friberg H, et al. Serum tau and neurological outcome in cardiac arrest. Ann Neurol. 2017;82(5):665–75.CrossRef Mattsson N, Zetterberg H, Nielsen N, Blennow K, Dankiewicz J, Friberg H, et al. Serum tau and neurological outcome in cardiac arrest. Ann Neurol. 2017;82(5):665–75.CrossRef
9.
go back to reference Kaneko T, Kasaoka S, Miyauchi T, Fujita M, Oda Y, Tsuruta R, et al. Serum glial fibrillary acidic protein as a predictive biomarker of neurological outcome after cardiac arrest. Resuscitation. 2009;80(7):790–4.CrossRef Kaneko T, Kasaoka S, Miyauchi T, Fujita M, Oda Y, Tsuruta R, et al. Serum glial fibrillary acidic protein as a predictive biomarker of neurological outcome after cardiac arrest. Resuscitation. 2009;80(7):790–4.CrossRef
10.
go back to reference Ebner F, Moseby-Knappe M, Mattsson-Carlgren N, Lilja G, Dragancea I, Unden J, et al. Serum GFAP and UCH-L1 for the prediction of neurological outcome in comatose cardiac arrest patients. Resuscitation. 2020. Ebner F, Moseby-Knappe M, Mattsson-Carlgren N, Lilja G, Dragancea I, Unden J, et al. Serum GFAP and UCH-L1 for the prediction of neurological outcome in comatose cardiac arrest patients. Resuscitation. 2020.
11.
go back to reference Ok G, Aydin D, Erbuyun K, Gursoy C, Taneli F, Bilge S, et al. Neurological outcome after cardiac arrest: a prospective study of the predictive ability of prognostic biomarkers neuron-specific enolase, glial fibrillary acidic protein, S-100B, and procalcitonin. Turk J Med Sci. 2016;46(5):1459–68.CrossRef Ok G, Aydin D, Erbuyun K, Gursoy C, Taneli F, Bilge S, et al. Neurological outcome after cardiac arrest: a prospective study of the predictive ability of prognostic biomarkers neuron-specific enolase, glial fibrillary acidic protein, S-100B, and procalcitonin. Turk J Med Sci. 2016;46(5):1459–68.CrossRef
12.
go back to reference Annborn M, Nilsson F, Dankiewicz J, Rundgren M, Hertel S, Struck J, et al. The combination of biomarkers for prognostication of long-term outcome in patients treated with mild hypothermia after out-of-hospital cardiac arrest-a pilot study. Ther Hypothermia Temp Manag. 2016;6(2):85–90.CrossRef Annborn M, Nilsson F, Dankiewicz J, Rundgren M, Hertel S, Struck J, et al. The combination of biomarkers for prognostication of long-term outcome in patients treated with mild hypothermia after out-of-hospital cardiac arrest-a pilot study. Ther Hypothermia Temp Manag. 2016;6(2):85–90.CrossRef
13.
go back to reference Düring J, Annborn M, Cronberg T, Dankiewicz J, Devaux Y, Hassager C, et al. Copeptin as a marker of outcome after cardiac arrest: a sub-study of the TTM trial. Critical care (London, England). 2020;24(1):185-. Düring J, Annborn M, Cronberg T, Dankiewicz J, Devaux Y, Hassager C, et al. Copeptin as a marker of outcome after cardiac arrest: a sub-study of the TTM trial. Critical care (London, England). 2020;24(1):185-.
14.
go back to reference Myhre PL, Tiainen M, Pettila V, Vaahersalo J, Hagve TA, Kurola J, et al. NT-proBNP in patients with out-of-hospital cardiac arrest: Results from the FINNRESUSCI Study. Resuscitation. 2016;104:12–8.CrossRef Myhre PL, Tiainen M, Pettila V, Vaahersalo J, Hagve TA, Kurola J, et al. NT-proBNP in patients with out-of-hospital cardiac arrest: Results from the FINNRESUSCI Study. Resuscitation. 2016;104:12–8.CrossRef
15.
go back to reference Bro-Jeppesen J, Kjaergaard J, Stammet P, Wise MP, Hovdenes J, Åneman A, et al. Predictive value of interleukin-6 in post-cardiac arrest patients treated with targeted temperature management at 33 °C or 36 °C. Resuscitation. 2016;98:1–8.CrossRef Bro-Jeppesen J, Kjaergaard J, Stammet P, Wise MP, Hovdenes J, Åneman A, et al. Predictive value of interleukin-6 in post-cardiac arrest patients treated with targeted temperature management at 33 °C or 36 °C. Resuscitation. 2016;98:1–8.CrossRef
16.
go back to reference Frydland M, Kjaergaard J, Erlinge D, Stammet P, Nielsen N, Wanscher M, et al. Usefulness of serum B-type natriuretic peptide levels in comatose patients resuscitated from out-of-hospital cardiac arrest to predict outcome. Am J Cardiol. 2016;118(7):998–1005.CrossRef Frydland M, Kjaergaard J, Erlinge D, Stammet P, Nielsen N, Wanscher M, et al. Usefulness of serum B-type natriuretic peptide levels in comatose patients resuscitated from out-of-hospital cardiac arrest to predict outcome. Am J Cardiol. 2016;118(7):998–1005.CrossRef
17.
go back to reference Holmgren G, Andersson P, Jakobsson A, Frigyesi A. Artificial neural networks improve and simplify intensive care mortality prognostication: a national cohort study of 217,289 first-time intensive care unit admissions. J Intensive Care. 2019;7:44.CrossRef Holmgren G, Andersson P, Jakobsson A, Frigyesi A. Artificial neural networks improve and simplify intensive care mortality prognostication: a national cohort study of 217,289 first-time intensive care unit admissions. J Intensive Care. 2019;7:44.CrossRef
18.
go back to reference Majkowska A, Mittal S, Steiner DF, Reicher JJ, McKinney SM, Duggan GE, et al. Chest radiograph interpretation with deep learning models: assessment with radiologist-adjudicated reference standards and population-adjusted evaluation. Radiology. 2020;294(2):421–31.CrossRef Majkowska A, Mittal S, Steiner DF, Reicher JJ, McKinney SM, Duggan GE, et al. Chest radiograph interpretation with deep learning models: assessment with radiologist-adjudicated reference standards and population-adjusted evaluation. Radiology. 2020;294(2):421–31.CrossRef
19.
go back to reference Kwon JM, Jeon KH, Kim HM, Kim MJ, Lim S, Kim KH, et al. Deep-learning-based out-of-hospital cardiac arrest prognostic system to predict clinical outcomes. Resuscitation. 2019;139:84–91.CrossRef Kwon JM, Jeon KH, Kim HM, Kim MJ, Lim S, Kim KH, et al. Deep-learning-based out-of-hospital cardiac arrest prognostic system to predict clinical outcomes. Resuscitation. 2019;139:84–91.CrossRef
20.
go back to reference Johnsson J, Bjornsson O, Andersson P, Jakobsson A, Cronberg T, Lilja G, et al. Artificial neural networks improve early outcome prediction and risk classification in out-of-hospital cardiac arrest patients admitted to intensive care. Crit Care. 2020;24(1):474.CrossRef Johnsson J, Bjornsson O, Andersson P, Jakobsson A, Cronberg T, Lilja G, et al. Artificial neural networks improve early outcome prediction and risk classification in out-of-hospital cardiac arrest patients admitted to intensive care. Crit Care. 2020;24(1):474.CrossRef
21.
go back to reference Nielsen N, Wetterslev J, Cronberg T, Erlinge D, Gasche Y, Hassager C, et al. Targeted temperature management at 33 degrees C versus 36 degrees C after cardiac arrest. N Engl J Med. 2013;369(23):2197–206.CrossRef Nielsen N, Wetterslev J, Cronberg T, Erlinge D, Gasche Y, Hassager C, et al. Targeted temperature management at 33 degrees C versus 36 degrees C after cardiac arrest. N Engl J Med. 2013;369(23):2197–206.CrossRef
22.
go back to reference Nielsen N, Wetterslev J, Al-Subaie N, Andersson B, Bro-Jeppesen J, Bishop G, et al. Target temperature management after out-of-hospital cardiac arrest–a randomized, parallel-group, assessor-blinded clinical trial–rationale and design. Am Heart J. 2012;163(4):541–8.CrossRef Nielsen N, Wetterslev J, Al-Subaie N, Andersson B, Bro-Jeppesen J, Bishop G, et al. Target temperature management after out-of-hospital cardiac arrest–a randomized, parallel-group, assessor-blinded clinical trial–rationale and design. Am Heart J. 2012;163(4):541–8.CrossRef
23.
go back to reference Nielsen N, Winkel P, Cronberg T, Erlinge D, Friberg H, Gasche Y, et al. Detailed statistical analysis plan for the target temperature management after out-of-hospital cardiac arrest trial. Trials. 2013;14:300.CrossRef Nielsen N, Winkel P, Cronberg T, Erlinge D, Friberg H, Gasche Y, et al. Detailed statistical analysis plan for the target temperature management after out-of-hospital cardiac arrest trial. Trials. 2013;14:300.CrossRef
24.
go back to reference Cronberg T, Lilja G, Horn J, Kjaergaard J, Wise MP, Pellis T, et al. Neurologic function and health-related quality of life in patients following targeted temperature management at 33 degrees C vs 36 degrees C after out-of-hospital cardiac arrest: a randomized clinical trial. JAMA Neurol. 2015;72(6):634–41.CrossRef Cronberg T, Lilja G, Horn J, Kjaergaard J, Wise MP, Pellis T, et al. Neurologic function and health-related quality of life in patients following targeted temperature management at 33 degrees C vs 36 degrees C after out-of-hospital cardiac arrest: a randomized clinical trial. JAMA Neurol. 2015;72(6):634–41.CrossRef
25.
go back to reference A randomized clinical study of cardiopulmonary-cerebral resuscitation: design, methods, and patient characteristics. Brain Resuscitation Clinical Trial I Study Group. Am J Emerg Med. 1986;4(1):72–86. A randomized clinical study of cardiopulmonary-cerebral resuscitation: design, methods, and patient characteristics. Brain Resuscitation Clinical Trial I Study Group. Am J Emerg Med. 1986;4(1):72–86.
27.
go back to reference R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing; 2020. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing; 2020.
28.
go back to reference Python Core Team. Python: A dynamic, open source programming language. Python version 3.7 ed: Python Software Foundation; 2020. Python Core Team. Python: A dynamic, open source programming language. Python version 3.7 ed: Python Software Foundation; 2020.
29.
go back to reference Abadi M, Barham P, Chen JM, Chen ZF, Davis A, Dean J, et al. TensorFlow: a system for large-scale machine learning. In: Proceedings of Osdi'16: 12th Usenix symposium on operating systems design and implementation. 2016:265–83. Abadi M, Barham P, Chen JM, Chen ZF, Davis A, Dean J, et al. TensorFlow: a system for large-scale machine learning. In: Proceedings of Osdi'16: 12th Usenix symposium on operating systems design and implementation. 2016:265–83.
30.
go back to reference Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. Proceedings of the 31st international conference on neural information processing systems; Long Beach, California, USA: Curran Associates Inc.; 2017. p. 4768–77. Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. Proceedings of the 31st international conference on neural information processing systems; Long Beach, California, USA: Curran Associates Inc.; 2017. p. 4768–77.
31.
go back to reference Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.CrossRef Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.CrossRef
32.
go back to reference López-Ratón M, Rodríguez-Álvarez MX, Suárez CC, Sampedro FG. OptimalCutpoints: an R package for selecting optimal cutpoints in diagnostic tests. J Stat Softw. 2014;61(8):1–36.CrossRef López-Ratón M, Rodríguez-Álvarez MX, Suárez CC, Sampedro FG. OptimalCutpoints: an R package for selecting optimal cutpoints in diagnostic tests. J Stat Softw. 2014;61(8):1–36.CrossRef
33.
go back to reference Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Int Med. 2015;162(1):55–63.CrossRef Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Int Med. 2015;162(1):55–63.CrossRef
34.
go back to reference Moseby-Knappe M, Westhall E, Backman S, Mattsson-Carlgren N, Dragancea I, Lybeck A, et al. Performance of a guideline-recommended algorithm for prognostication of poor neurological outcome after cardiac arrest. Intensive Care Med. 2020. Moseby-Knappe M, Westhall E, Backman S, Mattsson-Carlgren N, Dragancea I, Lybeck A, et al. Performance of a guideline-recommended algorithm for prognostication of poor neurological outcome after cardiac arrest. Intensive Care Med. 2020.
35.
go back to reference Stammet P, Collignon O, Hassager C, Wise MP, Hovdenes J, Aneman A, et al. Neuron-specific enolase as a predictor of death or poor neurological outcome after out-of-hospital cardiac arrest and targeted temperature management at 33 degrees C and 36 degrees C. J Am Coll Cardiol. 2015;65(19):2104–14.CrossRef Stammet P, Collignon O, Hassager C, Wise MP, Hovdenes J, Aneman A, et al. Neuron-specific enolase as a predictor of death or poor neurological outcome after out-of-hospital cardiac arrest and targeted temperature management at 33 degrees C and 36 degrees C. J Am Coll Cardiol. 2015;65(19):2104–14.CrossRef
36.
go back to reference Martinell L, Nielsen N, Herlitz J, Karlsson T, Horn J, Wise MP, et al. Early predictors of poor outcome after out-of-hospital cardiac arrest. Critical care (London, England). 2017;21(1):96-. Martinell L, Nielsen N, Herlitz J, Karlsson T, Horn J, Wise MP, et al. Early predictors of poor outcome after out-of-hospital cardiac arrest. Critical care (London, England). 2017;21(1):96-.
37.
go back to reference Pareek N, Kordis P, Beckley-Hoelscher N, Pimenta D, Kocjancic ST, Jazbec A, et al. A practical risk score for early prediction of neurological outcome after out-of-hospital cardiac arrest: MIRACLE2. Eur Heart J. 2020. Pareek N, Kordis P, Beckley-Hoelscher N, Pimenta D, Kocjancic ST, Jazbec A, et al. A practical risk score for early prediction of neurological outcome after out-of-hospital cardiac arrest: MIRACLE2. Eur Heart J. 2020.
Metadata
Title
Predicting neurological outcome after out-of-hospital cardiac arrest with cumulative information; development and internal validation of an artificial neural network algorithm
Authors
Peder Andersson
Jesper Johnsson
Ola Björnsson
Tobias Cronberg
Christian Hassager
Henrik Zetterberg
Pascal Stammet
Johan Undén
Jesper Kjaergaard
Hans Friberg
Kaj Blennow
Gisela Lilja
Matt P. Wise
Josef Dankiewicz
Niklas Nielsen
Attila Frigyesi
Publication date
01-12-2021
Publisher
BioMed Central
Published in
Critical Care / Issue 1/2021
Electronic ISSN: 1364-8535
DOI
https://doi.org/10.1186/s13054-021-03505-9

Other articles of this Issue 1/2021

Critical Care 1/2021 Go to the issue