Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Medical Informatics and Decision Making
Published in: BMC Medical Informatics and Decision Making 1/2023

Open Access 01-12-2023 | Stroke | Research

A comparison of random survival forest and Cox regression for prediction of mortality in patients with hemorrhagic stroke

Authors: Yuxin Wang, Yuhan Deng, Yinliang Tan, Meihong Zhou, Yong Jiang, Baohua Liu

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Login to get access

Abstract

Objective

To evaluate RSF and Cox models for mortality prediction of hemorrhagic stroke (HS) patients in intensive care unit (ICU).

Methods

In the training set, the optimal models were selected using five-fold cross-validation and grid search method. In the test set, the bootstrap method was used to validate. The area under the curve(AUC) was used for discrimination, Brier Score (BS) was used for calibration, positive predictive value(PPV), negative predictive value(NPV), and F1 score were combined to compare.

Results

A total of 2,990 HS patients were included. For predicting the 7-day mortality, the mean AUCs for RSF and Cox regression were 0.875 and 0.761, while the mean BS were 0.083 and 0.108. For predicting the 28-day mortality, the mean AUCs for RSF and Cox regression were 0.794 and 0.649, while the mean BS were 0.129 and 0.174. The mean AUCs of RSF and Cox versus conventional scores for predicting patients’ 7-day mortality were 0.875 (RSF), 0.761 (COX), 0.736 (SAPS II), 0.723 (OASIS), 0.632 (SIRS), and 0.596 (SOFA), respectively.

Conclusions

RSF provided a better clinical reference than Cox. Creatine, temperature, anion gap and sodium were important variables in both models.
Appendix
Available only for authorised users
Literature
1.
Thayabaranathan T, Kim J, Cadilhac DA, Thrift AG, Donnan GA, Howard G, Howard VJ, Rothwell PM, Feigin V, Norrving B et al. Global stroke statistics 2022. Int J Stroke 2022.
2.
Feigin VL, Stark BA, Johnson CO, Roth GA, Bisignano C, Abady GG, Abbasifard M, Abbasi-Kangevari M, Abd-Allah F, Abedi V, et al. Global, regional, and national burden of stroke and its risk factors, 1990–2019: a systematic analysis for the global burden of Disease Study 2019. Lancet Neurol. 2021;20(10):795–820.
3.
Akyea RK, Georgiopoulos G, Iyen B, Kai J, Qureshi N, Ntaios G. Comparison of risk of Serious Cardiovascular events after hemorrhagic versus ischemic stroke: a Population-Based study. Thromb Haemost. 2022;122(11):1921–31.PubMed
4.
van Asch CJ, Luitse MJ, Rinkel GJ, van der Tweel I, Algra A, Klijn CJ. Incidence, case fatality, and functional outcome of intracerebral haemorrhage over time, according to age, sex, and ethnic origin: a systematic review and meta-analysis. Lancet Neurol. 2010;9(2):167–76.PubMed
5.
Carval T, Garret C, Guillon B, Lascarrou JB, Martin M, Lemarie J, Dupeyrat J, Seguin A, Zambon O, Reignier J, et al. Outcomes of patients admitted to the ICU for acute stroke: a retrospective cohort. Bmc Anesthesiol. 2022;22(1):235.PubMedPubMedCentral
6.
Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.PubMed
7.
Ferreira FL, Bota DP, Bross A, Melot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286(14):1754–8.PubMed
8.
Johnson AE, Kramer AA, Clifford GD. A new severity of illness scale using a subset of Acute Physiology and Chronic Health evaluation data elements shows comparable predictive accuracy. Crit Care Med. 2013;41(7):1711–8.PubMed
9.
Le Gall JR, Lemeshow S, Saulnier F. A new simplified Acute Physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–63.PubMed
10.
Rahmatinejad Z, Hoseini B, Rahmatinejad F, Abu-Hanna A, Bergquist R, Pourmand A, Miri M, Eslami S. Internal validation of the Predictive performance of Models based on three ED and ICU Scoring Systems to predict Inhospital Mortality for Intensive Care Patients referred from the Emergency Department. Biomed Res Int. 2022;2022:3964063.PubMedPubMedCentral
11.
Afrash MR, Mirbagheri E, Mashoufi M, Kazemi-Arpanahi H. Optimizing prognostic factors of five-year survival in gastric cancer patients using feature selection techniques with machine learning algorithms: a comparative study. BMC Med Inform Decis Mak. 2023;23(1):54.PubMedPubMedCentral
12.
Deng Y, Liu S, Wang Z, Wang Y, Jiang Y, Liu B. Explainable time-series deep learning models for the prediction of mortality, prolonged length of stay and 30-day readmission in intensive care patients. Front Med. 2022;9:933037.
13.
Ho WM, Lin JR, Wang HH, Liou CW, Chang KC, Lee JD, Peng TY, Yang JT, Chang YJ, Chang CH, et al. Prediction of in-hospital stroke mortality in critical care unit. SpringerPlus. 2016;5(1):1051.PubMedPubMedCentral
14.
Smith EE, Shobha N, Dai D, Olson DM, Reeves MJ, Saver JL, Hernandez AF, Peterson ED, Fonarow GC, Schwamm LH. A risk score for in-hospital death in patients admitted with ischemic or hemorrhagic stroke. J Am Heart Association. 2013;2(1):e005207.
15.
Van Calster B, Wynants L. Machine learning in Medicine. N Engl J Med. 2019;380(26):2588.PubMed
16.
Steele AJ, Denaxas SC, Shah AD, Hemingway H, Luscombe NM. Machine learning models in electronic health records can outperform conventional survival models for predicting patient mortality in coronary artery disease. PLoS ONE. 2018;13(8):e0202344.PubMedPubMedCentral
17.
Sabetian G, Azimi A, Kazemi A, Hoseini B, Asmarian N, Khaloo V, Zand F, Masjedi M, Shahriarirad R, Shahriarirad S. Prediction of patients with COVID-19 requiring Intensive Care: a cross-sectional study based on machine-learning Approach from Iran. Indian J Crit care Medicine: peer-reviewed Official Publication Indian Soc Crit Care Med. 2022;26(6):688–95.
18.
Lin CH, Hsu KC, Johnson KR, Fann YC, Tsai CH, Sun Y, Lien LM, Chang WL, Chen PL, Lin CL, et al. Evaluation of machine learning methods to stroke outcome prediction using a nationwide disease registry. Comput Methods Programs Biomed. 2020;190:105381.PubMedPubMedCentral
19.
Trevisi G, Caccavella VM, Scerrati A, Signorelli F, Salamone GG, Orsini K, Fasciani C, D’Arrigo S, Auricchio AM, D’Onofrio G, et al. Machine learning model prediction of 6-month functional outcome in elderly patients with intracerebral hemorrhage. Neurosurg Rev. 2022;45(4):2857–67.PubMedPubMedCentral
20.
Jiang L, Zhou L, Yong W, Cui J, Geng W, Chen H, Zou J, Chen Y, Yin X, Chen YC. A deep learning-based model for prediction of hemorrhagic transformation after stroke. Brain Pathol. 2023;33(2):e13023.PubMed
21.
Lee KS, Jang JY, Yu YD, Heo JS, Han HS, Yoon YS, Kang CM, Hwang HK, Kang S. Usefulness of artificial intelligence for predicting recurrence following surgery for pancreatic cancer: retrospective cohort study. Int J Surg. 2021;93:106050.PubMed
22.
Zhang L, Huang T, Xu F, Li S, Zheng S, Lyu J, Yin H. Prediction of prognosis in elderly patients with sepsis based on machine learning (random survival forest). BMC Emerg Med. 2022;22(1):26.PubMedPubMedCentral
23.
Xiao J, Mo M, Wang Z, Zhou C, Shen J, Yuan J, He Y, Zheng Y. The application and comparison of machine learning models for the prediction of breast Cancer prognosis: Retrospective Cohort Study. JMIR Med Inform. 2022;10(2):e33440.PubMedPubMedCentral
24.
Bao L, Wang YT, Zhuang JL, Liu AJ, Dong YJ, Chu B, Chen XH, Lu MQ, Shi L, Gao S, et al. Machine learning-based overall survival prediction of Elderly patients with multiple myeloma from Multicentre Real-Life Data. Front Oncol. 2022;12:922039.PubMedPubMedCentral
25.
Grendas LN, Chiapella L, Rodante DE, Daray FM. Comparison of traditional model-based statistical methods with machine learning for the prediction of suicide behaviour. J Psychiatr Res. 2021;145:85–91.PubMed
26.
Pei W, Wang C, Liao H, Chen X, Wei Y, Huang X, Liang X, Bao H, Su D, Jin G. MRI-based random survival forest model improves prediction of progression-free survival to induction chemotherapy plus concurrent Chemoradiotherapy in Locoregionally Advanced nasopharyngeal carcinoma. BMC Cancer. 2022;22(1):739.PubMedPubMedCentral
27.
28.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594.PubMed
29.
Carlsson M, Wilsgaard T, Johnsen SH, Johnsen LH, Lochen ML, Njolstad I, Mathiesen EB. Long-term survival, causes of death, and Trends in 5-Year Mortality after Intracerebral Hemorrhage: the Tromso Study. Stroke. 2021;52(12):3883–90.PubMed
30.
Wang Y, Wang J, Chen S, Li B, Lu X, Li J. Different changing patterns for Stroke Subtype Mortality Attributable to High Sodium Intake in China during 1990 to 2019. Stroke. 2023;54(4):1078–87.PubMed
31.
Allison PD. Multiple imputation for missing data - A cautionary tale. Sociol Method Res. 2000;28(3):301–9.
32.
Yosefian I, Farkhani EM, Baneshi MR. Application of Random Forest Survival Models to increase generalizability of decision trees: a Case Study in Acute myocardial infarction. Comput Math Methods Med. 2015;2015:576413.PubMedPubMedCentral
33.
Mogensen UB, Ishwaran H, Gerds TA. Evaluating Random forests for Survival Analysis using Prediction Error Curves. J Stat Softw. 2012;50(11):1–23.PubMedPubMedCentral
34.
Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random Survival Forests. Wiley StatsRef: Statistics Reference Online 2019.
35.
Wang X, Gong G, Li N, Qiu S. Detection analysis of epileptic EEG using a Novel Random Forest Model Combined with Grid Search optimization. Front Hum Neurosci. 2019;13:52.PubMedPubMedCentral
36.
Ishwaran H. Variable importance in binary regression trees and forests. Electron J Stat. 2007;1:519–37.
37.
Kay R. Goodness of fit methods for the proportional hazards regression model: a review. Rev Epidemiol Sante Publique. 1984;32(3–4):185–98.PubMed
38.
Grunkemeier GL, Wu Y. Bootstrap resampling methods: something for nothing? Ann Thorac Surg. 2004;77(4):1142–4.PubMed
39.
Murphy AH. A New Decomposition of the Brier score - formulation and interpretation. Mon Weather Rev. 1986;114(12):2671–3.
40.
Tang H, Jin Z, Deng J, She Y, Zhong Y, Sun W, Ren Y, Cao N, Chen C. Development and validation of a deep learning model to predict the survival of patients in ICU. J Am Med Inform Assoc. 2022;29(9):1567–76.PubMedPubMedCentral
41.
Hadanny A, Shouval R, Wu J, Gale CP, Unger R, Zahger D, Gottlieb S, Matetzky S, Goldenberg I, Beigel R, et al. Machine learning-based prediction of 1-year mortality for acute coronary syndrome(). J Cardiol. 2022;79(3):342–51.PubMed
42.
Qiu X, Gao J, Yang J, Hu J, Hu W, Kong L, Lu JJ. A comparison study of machine learning (Random Survival Forest) and Classic Statistic (Cox Proportional Hazards) for Predicting Progression in High-Grade Glioma after Proton and Carbon Ion Radiotherapy. Front Oncol. 2020;10:551420.PubMedPubMedCentral
43.
Lin CH, Kuo YW, Huang YC, Lee M, Huang YW, Kuo CF, Lee JD. Development and validation of a Novel score for Predicting Long-Term Mortality after an Acute Ischemic Stroke. Int J Environ Res Public Health 2023, 20(4).
44.
Huang T, Huang L, Yang R, Li S, He N, Feng A, Li L, Lyu J. Machine learning models for predicting survival in patients with ampullary adenocarcinoma. Asia Pac J Oncol Nurs. 2022;9(12):100141.PubMedPubMedCentral
45.
Pickering JW, Frampton CM, Walker RJ, Shaw GM, Endre ZH. Four hour creatinine clearance is better than plasma creatinine for monitoring renal function in critically ill patients. Crit Care. 2012;16(3):R107.PubMedPubMedCentral
46.
Luo H, Yang X, Chen K, Lan S, Liao G, Xu J. Blood creatinine and urea nitrogen at ICU admission and the risk of in-hospital death and 1-year mortality in patients with intracranial hemorrhage. Front Cardiovasc Med. 2022;9:967614.PubMedPubMedCentral
47.
Liddle LJ, Dirks CA, Almekhlafi M, Colbourne F. An ambiguous role for fever in worsening Outcome after Intracerebral Hemorrhage. Transl Stroke Res. 2023;14(2):123–36.PubMed
48.
Iglesias-Rey R, Rodriguez-Yanez M, Arias S, Santamaria M, Rodriguez-Castro E, Lopez-Dequidt I, Hervella P, Sobrino T, Campos F, Castillo J. Inflammation, edema and poor outcome are associated with hyperthermia in hypertensive intracerebral hemorrhages. Eur J Neurol. 2018;25(9):1161–8.PubMedPubMedCentral
49.
Kraut JA, Nagami GT. The serum anion gap in the evaluation of acid-base disorders: what are its limitations and can its effectiveness be improved? Clin J Am Soc Nephrol. 2013;8(11):2018–24.PubMedPubMedCentral
50.
Liu X, Feng Y, Zhu X, Shi Y, Lin M, Song X, Tu J, Yuan E. Serum anion gap at admission predicts all-cause mortality in critically ill patients with cerebral infarction: evidence from the MIMIC-III database. Biomarkers: Biochem Indic Exposure Response Susceptibility Chemicals. 2020;25(8):725–32.
51.
Shen J, Li DL, Yang ZS, Zhang YZ, Li ZY. Anion gap predicts the long-term neurological and cognitive outcomes of spontaneous intracerebral hemorrhage. Eur Rev Med Pharmacol Sci. 2022;26(9):3230–6.PubMed
52.
Strazzullo P, D’Elia L, Kandala NB, Cappuccio FP. Salt intake, stroke, and cardiovascular disease: meta-analysis of prospective studies. BMJ. 2009;339:b4567.PubMedPubMedCentral
Metadata
Title
A comparison of random survival forest and Cox regression for prediction of mortality in patients with hemorrhagic stroke
Authors
Yuxin Wang
Yuhan Deng
Yinliang Tan
Meihong Zhou
Yong Jiang
Baohua Liu
Publication date
01-12-2023
Publisher
BioMed Central
Keyword
Stroke
Published in
BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-023-02293-2

Other articles of this Issue 1/2023

BMC Medical Informatics and Decision Making 1/2023 Go to the issue

Research

Descriptive forest: experiments on a novel tree-structure-generalization method for describing cardiovascular diseases

Research article

A study on predicting the length of hospital stay for Chinese patients with ischemic stroke based on the XGBoost algorithm

Research

On Scene Injury Severity Prediction (OSISP) model for trauma developed using the Swedish Trauma Registry

Research

Short-term prognostic models for severe acute kidney injury patients receiving prolonged intermittent renal replacement therapy based on machine learning

Research

Risk prediction of heart failure in patients with ischemic heart disease using network analytics and stacking ensemble learning

Research

Development and validation of predictive models for unplanned hospitalization in the Basque Country: analyzing the variability of non-deterministic algorithms