Skip to main content
Top
Published in: Critical Care 1/2020

01-12-2020 | Acute Kidney Injury | Research

Machine learning algorithm to predict mortality in patients undergoing continuous renal replacement therapy

Authors: Min Woo Kang, Jayoun Kim, Dong Ki Kim, Kook-Hwan Oh, Kwon Wook Joo, Yon Su Kim, Seung Seok Han

Published in: Critical Care | Issue 1/2020

Login to get access

Abstract

Background

Previous scoring models such as the Acute Physiologic Assessment and Chronic Health Evaluation II (APACHE II) and the Sequential Organ Failure Assessment (SOFA) scoring systems do not adequately predict mortality of patients undergoing continuous renal replacement therapy (CRRT) for severe acute kidney injury. Accordingly, the present study applies machine learning algorithms to improve prediction accuracy for this patient subset.

Methods

We randomly divided a total of 1571 adult patients who started CRRT for acute kidney injury into training (70%, n = 1094) and test (30%, n = 477) sets. The primary output consisted of the probability of mortality during admission to the intensive care unit (ICU) or hospital. We compared the area under the receiver operating characteristic curves (AUCs) of several machine learning algorithms with that of the APACHE II, SOFA, and the new abbreviated mortality scoring system for acute kidney injury with CRRT (MOSAIC model) results.

Results

For the ICU mortality, the random forest model showed the highest AUC (0.784 [0.744–0.825]), and the artificial neural network and extreme gradient boost models demonstrated the next best results (0.776 [0.735–0.818]). The AUC of the random forest model was higher than 0.611 (0.583–0.640), 0.677 (0.651–0.703), and 0.722 (0.677–0.767), as achieved by APACHE II, SOFA, and MOSAIC, respectively. The machine learning models also predicted in-hospital mortality better than APACHE II, SOFA, and MOSAIC.

Conclusion

Machine learning algorithms increase the accuracy of mortality prediction for patients undergoing CRRT for acute kidney injury compared with previous scoring models.
Appendix
Available only for authorised users
Literature
1.
go back to reference Odutayo A, Wong CX, Farkouh M, Altman DG, Hopewell S, Emdin CA, Hunn BH. AKI and long-term risk for cardiovascular events and mortality. J Am Soc Nephrol. 2017;28(1):377–87.PubMedCrossRef Odutayo A, Wong CX, Farkouh M, Altman DG, Hopewell S, Emdin CA, Hunn BH. AKI and long-term risk for cardiovascular events and mortality. J Am Soc Nephrol. 2017;28(1):377–87.PubMedCrossRef
2.
go back to reference Susantitaphong P, Cruz DN, Cerda J, Abulfaraj M, Alqahtani F, Koulouridis I, Jaber BL. Acute Kidney Injury Advisory Group of the American Society of N: world incidence of AKI: a meta-analysis. Clin J Am Soc Nephrol. 2013;8(9):1482–93.PubMedPubMedCentralCrossRef Susantitaphong P, Cruz DN, Cerda J, Abulfaraj M, Alqahtani F, Koulouridis I, Jaber BL. Acute Kidney Injury Advisory Group of the American Society of N: world incidence of AKI: a meta-analysis. Clin J Am Soc Nephrol. 2013;8(9):1482–93.PubMedPubMedCentralCrossRef
3.
go back to reference Bellomo R. The epidemiology of acute renal failure: 1975 versus 2005. Curr Opin Crit Care. 2006;12(6):557–60.PubMedCrossRef Bellomo R. The epidemiology of acute renal failure: 1975 versus 2005. Curr Opin Crit Care. 2006;12(6):557–60.PubMedCrossRef
4.
go back to reference Uchino S. The epidemiology of acute renal failure in the world. Curr Opin Crit Care. 2006;12(6):538–43.PubMedCrossRef Uchino S. The epidemiology of acute renal failure in the world. Curr Opin Crit Care. 2006;12(6):538–43.PubMedCrossRef
5.
go back to reference Waikar SS, Liu KD, Chertow GM. Diagnosis, epidemiology and outcomes of acute kidney injury. Clin J Am Soc Nephrol. 2008;3(3):844–61.PubMedCrossRef Waikar SS, Liu KD, Chertow GM. Diagnosis, epidemiology and outcomes of acute kidney injury. Clin J Am Soc Nephrol. 2008;3(3):844–61.PubMedCrossRef
6.
go back to reference Hoste EA, Bagshaw SM, Bellomo R, Cely CM, Colman R, Cruz DN, Edipidis K, Forni LG, Gomersall CD, Govil D, et al. Epidemiology of acute kidney injury in critically ill patients: the multinational AKI-EPI study. Intensive Care Med. 2015;41(8):1411–23.PubMedCrossRef Hoste EA, Bagshaw SM, Bellomo R, Cely CM, Colman R, Cruz DN, Edipidis K, Forni LG, Gomersall CD, Govil D, et al. Epidemiology of acute kidney injury in critically ill patients: the multinational AKI-EPI study. Intensive Care Med. 2015;41(8):1411–23.PubMedCrossRef
7.
go back to reference Tolwani A. Continuous renal-replacement therapy for acute kidney injury. N Engl J Med. 2012;367(26):2505–14.PubMedCrossRef Tolwani A. Continuous renal-replacement therapy for acute kidney injury. N Engl J Med. 2012;367(26):2505–14.PubMedCrossRef
8.
go back to reference Liu KD, Himmelfarb J, Paganini E, Ikizler TA, Soroko SH, Mehta RL, Chertow GM. Timing of initiation of dialysis in critically ill patients with acute kidney injury. Clin J Am Soc Nephrol. 2006;1(5):915–9.PubMedCrossRef Liu KD, Himmelfarb J, Paganini E, Ikizler TA, Soroko SH, Mehta RL, Chertow GM. Timing of initiation of dialysis in critically ill patients with acute kidney injury. Clin J Am Soc Nephrol. 2006;1(5):915–9.PubMedCrossRef
9.
go back to reference Saudan P, Niederberger M, De Seigneux S, Romand J, Pugin J, Perneger T, Martin PY. Adding a dialysis dose to continuous hemofiltration increases survival in patients with acute renal failure. Kidney Int. 2006;70(7):1312–7.PubMedCrossRef Saudan P, Niederberger M, De Seigneux S, Romand J, Pugin J, Perneger T, Martin PY. Adding a dialysis dose to continuous hemofiltration increases survival in patients with acute renal failure. Kidney Int. 2006;70(7):1312–7.PubMedCrossRef
10.
go back to reference Uchino S, Kellum JA, Bellomo R, Doig GS, Morimatsu H, Morgera S, Schetz M, Tan I, Bouman C, Macedo E, et al. Acute renal failure in critically ill patients: a multinational, multicenter study. JAMA. 2005;294(7):813–8.PubMedCrossRef Uchino S, Kellum JA, Bellomo R, Doig GS, Morimatsu H, Morgera S, Schetz M, Tan I, Bouman C, Macedo E, et al. Acute renal failure in critically ill patients: a multinational, multicenter study. JAMA. 2005;294(7):813–8.PubMedCrossRef
11.
go back to reference Demirjian S, Chertow GM, Zhang JH, O'Connor TZ, Vitale J, Paganini EP, Palevsky PM, Network VNARFT. Model to predict mortality in critically ill adults with acute kidney injury. Clin J Am Soc Nephrol. 2011;6(9):2114–20.PubMedPubMedCentralCrossRef Demirjian S, Chertow GM, Zhang JH, O'Connor TZ, Vitale J, Paganini EP, Palevsky PM, Network VNARFT. Model to predict mortality in critically ill adults with acute kidney injury. Clin J Am Soc Nephrol. 2011;6(9):2114–20.PubMedPubMedCentralCrossRef
12.
go back to reference da Hora PR, Ramos JG, Mendonca EJ, Miranda EA, Dutra FR, Coelho MF, Pedroza AC, Correia LC, Batista PB, Macedo E, et al. A clinical score to predict mortality in septic acute kidney injury patients requiring continuous renal replacement therapy: the HELENICC score. BMC Anesthesiol. 2017;17(1):21.CrossRef da Hora PR, Ramos JG, Mendonca EJ, Miranda EA, Dutra FR, Coelho MF, Pedroza AC, Correia LC, Batista PB, Macedo E, et al. A clinical score to predict mortality in septic acute kidney injury patients requiring continuous renal replacement therapy: the HELENICC score. BMC Anesthesiol. 2017;17(1):21.CrossRef
13.
go back to reference Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.PubMedCrossRef Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.PubMedCrossRef
14.
go back to reference Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, Reinhart CK, Suter PM, Thijs LG. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–10.PubMedCrossRef Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, Reinhart CK, Suter PM, Thijs LG. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–10.PubMedCrossRef
17.
go back to reference Yang F, Wang HZ, Mi H, Lin CD, Cai WW. Using random forest for reliable classification and cost-sensitive learning for medical diagnosis. BMC Bioinformatics. 2009;10(Suppl 1):S22.PubMedPubMedCentralCrossRef Yang F, Wang HZ, Mi H, Lin CD, Cai WW. Using random forest for reliable classification and cost-sensitive learning for medical diagnosis. BMC Bioinformatics. 2009;10(Suppl 1):S22.PubMedPubMedCentralCrossRef
18.
go back to reference Hsieh MH, Hsieh MJ, Chen CM, Hsieh CC, Chao CM, Lai CC. An artificial neural network model for predicting successful extubation in intensive care units. J Clin Med. 2018;7(9):240.PubMedCentralCrossRef Hsieh MH, Hsieh MJ, Chen CM, Hsieh CC, Chao CM, Lai CC. An artificial neural network model for predicting successful extubation in intensive care units. J Clin Med. 2018;7(9):240.PubMedCentralCrossRef
19.
go back to reference Flechet M, Falini S, Bonetti C, Guiza F, Schetz M, Van den Berghe G, Meyfroidt G. Machine learning versus physicians’ prediction of acute kidney injury in critically ill adults: a prospective evaluation of the AKI predictor. Crit Care. 2019;23(1):282.PubMedPubMedCentralCrossRef Flechet M, Falini S, Bonetti C, Guiza F, Schetz M, Van den Berghe G, Meyfroidt G. Machine learning versus physicians’ prediction of acute kidney injury in critically ill adults: a prospective evaluation of the AKI predictor. Crit Care. 2019;23(1):282.PubMedPubMedCentralCrossRef
20.
go back to reference Shillan D, Sterne JAC, Champneys A, Gibbison B. Use of machine learning to analyse routinely collected intensive care unit data: a systematic review. Crit Care. 2019;23(1):284.PubMedPubMedCentralCrossRef Shillan D, Sterne JAC, Champneys A, Gibbison B. Use of machine learning to analyse routinely collected intensive care unit data: a systematic review. Crit Care. 2019;23(1):284.PubMedPubMedCentralCrossRef
21.
go back to reference Zhang Z, Ho KM, Hong Y. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care. 2019;23(1):112.PubMedPubMedCentralCrossRef Zhang Z, Ho KM, Hong Y. Machine learning for the prediction of volume responsiveness in patients with oliguric acute kidney injury in critical care. Crit Care. 2019;23(1):112.PubMedPubMedCentralCrossRef
22.
go back to reference Kim Y, Park N, Kim J, Kim DK, Chin HJ, Na KY, Joo KW, Kim YS, Kim S, Han SS. Development of a new mortality scoring system for acute kidney injury with continuous renal replacement therapy. Nephrology (Carlton). 2019;24(12):1233–40.CrossRef Kim Y, Park N, Kim J, Kim DK, Chin HJ, Na KY, Joo KW, Kim YS, Kim S, Han SS. Development of a new mortality scoring system for acute kidney injury with continuous renal replacement therapy. Nephrology (Carlton). 2019;24(12):1233–40.CrossRef
23.
go back to reference Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26(6):565–74.CrossRef Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26(6):565–74.CrossRef
24.
25.
go back to reference Arts DG, de Keizer NF, Vroom MB, de Jonge E. Reliability and accuracy of Sequential Organ Failure Assessment (SOFA) scoring. Crit Care Med. 2005;33(9):1988–93.PubMedCrossRef Arts DG, de Keizer NF, Vroom MB, de Jonge E. Reliability and accuracy of Sequential Organ Failure Assessment (SOFA) scoring. Crit Care Med. 2005;33(9):1988–93.PubMedCrossRef
26.
go back to reference Asai N, Watanabe H, Shiota A, Kato H, Sakanashi D, Hagihara M, Koizumi Y, Yamagishi Y, Suematsu H, Mikamo H. Efficacy and accuracy of qSOFA and SOFA scores as prognostic tools for community-acquired and healthcare-associated pneumonia. Int J Infect Dis. 2019;84:89–96.PubMedCrossRef Asai N, Watanabe H, Shiota A, Kato H, Sakanashi D, Hagihara M, Koizumi Y, Yamagishi Y, Suematsu H, Mikamo H. Efficacy and accuracy of qSOFA and SOFA scores as prognostic tools for community-acquired and healthcare-associated pneumonia. Int J Infect Dis. 2019;84:89–96.PubMedCrossRef
27.
go back to reference Jones AE, Trzeciak S, Kline JA. The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med. 2009;37(5):1649–54.PubMedPubMedCentralCrossRef Jones AE, Trzeciak S, Kline JA. The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med. 2009;37(5):1649–54.PubMedPubMedCentralCrossRef
28.
go back to reference Sawicka W, Owczuk R, Wujtewicz MA, Wujtewicz M. The effectiveness of the APACHE II, SAPS II and SOFA prognostic scoring systems in patients with haematological malignancies in the intensive care unit. Anaesthesiol Intensive Ther. 2014;46(3):166–70.PubMedCrossRef Sawicka W, Owczuk R, Wujtewicz MA, Wujtewicz M. The effectiveness of the APACHE II, SAPS II and SOFA prognostic scoring systems in patients with haematological malignancies in the intensive care unit. Anaesthesiol Intensive Ther. 2014;46(3):166–70.PubMedCrossRef
29.
go back to reference Prasad B, Urbanski M, Ferguson TW, Karreman E, Tangri N. Early mortality on continuous renal replacement therapy (CRRT): the prairie CRRT study. Can J Kidney Health Dis. 2016;3:36.PubMedPubMedCentralCrossRef Prasad B, Urbanski M, Ferguson TW, Karreman E, Tangri N. Early mortality on continuous renal replacement therapy (CRRT): the prairie CRRT study. Can J Kidney Health Dis. 2016;3:36.PubMedPubMedCentralCrossRef
30.
go back to reference Lee S, Lee Y, Jang H, Moon H, Kim DK, Han SS. Heart rate is associated with mortality in patients undergoing continuous renal replacement therapy. Kidney Res Clin Pract. 2017;36(3):250–6.PubMedPubMedCentralCrossRef Lee S, Lee Y, Jang H, Moon H, Kim DK, Han SS. Heart rate is associated with mortality in patients undergoing continuous renal replacement therapy. Kidney Res Clin Pract. 2017;36(3):250–6.PubMedPubMedCentralCrossRef
31.
go back to reference Wong LS, Young JD. A comparison of ICU mortality prediction using the APACHE II scoring system and artificial neural networks. Anaesthesia. 1999;54(11):1048–54.PubMedCrossRef Wong LS, Young JD. A comparison of ICU mortality prediction using the APACHE II scoring system and artificial neural networks. Anaesthesia. 1999;54(11):1048–54.PubMedCrossRef
32.
go back to reference Nimgaonkar A, Karnad DR, Sudarshan S, Ohno-Machado L, Kohane I. Prediction of mortality in an Indian intensive care unit. Comparison between APACHE II and artificial neural networks. Intensive Care Med. 2004;30(2):248–53.PubMedCrossRef Nimgaonkar A, Karnad DR, Sudarshan S, Ohno-Machado L, Kohane I. Prediction of mortality in an Indian intensive care unit. Comparison between APACHE II and artificial neural networks. Intensive Care Med. 2004;30(2):248–53.PubMedCrossRef
33.
go back to reference Kim S, Kim W, Park RW. A comparison of intensive care unit mortality prediction models through the use of data mining techniques. Healthc Inform Res. 2011;17(4):232–43.PubMedPubMedCentralCrossRef Kim S, Kim W, Park RW. A comparison of intensive care unit mortality prediction models through the use of data mining techniques. Healthc Inform Res. 2011;17(4):232–43.PubMedPubMedCentralCrossRef
34.
go back to reference Goodfellow I, Bengio Y, Courville A. Deep learning: MIT Press; 2016:1–800. Goodfellow I, Bengio Y, Courville A. Deep learning: MIT Press; 2016:1–800.
Metadata
Title
Machine learning algorithm to predict mortality in patients undergoing continuous renal replacement therapy
Authors
Min Woo Kang
Jayoun Kim
Dong Ki Kim
Kook-Hwan Oh
Kwon Wook Joo
Yon Su Kim
Seung Seok Han
Publication date
01-12-2020
Publisher
BioMed Central
Published in
Critical Care / Issue 1/2020
Electronic ISSN: 1364-8535
DOI
https://doi.org/10.1186/s13054-020-2752-7

Other articles of this Issue 1/2020

Critical Care 1/2020 Go to the issue