Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2022

Open Access 01-12-2022 | Heart Surgery | Research

Prediction of acute kidney injury risk after cardiac surgery: using a hybrid machine learning algorithm

Authors: Yelena Petrosyan, Thierry G. Mesana, Louise Y. Sun

Published in: BMC Medical Informatics and Decision Making | Issue 1/2022

Login to get access

Abstract

Background

Acute kidney injury (AKI) is a serious complication after cardiac surgery. We derived and internally validated a Machine Learning preoperative model to predict cardiac surgery-associated AKI of any severity and compared its performance with parametric statistical models.

Methods

We conducted a retrospective study of adult patients who underwent major cardiac surgery requiring cardiopulmonary bypass between November 1st, 2009 and March 31st, 2015. AKI was defined according to the KDIGO criteria as stage 1 or greater, within 7 days of surgery. We randomly split the cohort into derivation and validation datasets. We developed three AKI risk models: (1) a hybrid machine learning (ML) algorithm, using Random Forests for variable selection, followed by high performance logistic regression; (2) a traditional logistic regression model and (3) an enhanced logistic regression model with 500 bootstraps, with backward variable selection. For each model, we assigned risk scores to each of the retained covariate and assessed model discrimination (C statistic) and calibration (Hosmer–Lemeshow goodness-of-fit test) in the validation datasets.

Results

Of 6522 included patients, 1760 (27.0%) developed AKI. The best performance was achieved by the hybrid ML algorithm to predict AKI of any severity. The ML and enhanced statistical models remained robust after internal validation (C statistic = 0.75; Hosmer–Lemeshow p = 0.804, and AUC = 0.74, Hosmer–Lemeshow p = 0.347, respectively).

Conclusions

We demonstrated that a hybrid ML model provides higher accuracy without sacrificing parsimony, computational efficiency, or interpretability, when compared with parametric statistical models. This score-based model can easily be used at the bedside to identify high-risk patients who may benefit from intensive perioperative monitoring and personalized management strategies.
Appendix
Available only for authorised users
Literature
1.
go back to reference Loef BG, Epema AH, Smilde TD, Henning RH, Ebels T, Navis G, et al. Immediate postoperative renal function deterioration in cardiac surgical patients predicts in-hospital mortality and long-term survival. J Am Soc Nephrol. 2005;16(1):195–200.PubMedCrossRef Loef BG, Epema AH, Smilde TD, Henning RH, Ebels T, Navis G, et al. Immediate postoperative renal function deterioration in cardiac surgical patients predicts in-hospital mortality and long-term survival. J Am Soc Nephrol. 2005;16(1):195–200.PubMedCrossRef
2.
go back to reference Mangano CM, Diamondstone LS, Ramsay JG, Aggarwal A, Herskowitz A, Mangano DT. Renal dysfunction after myocardial revascularization: risk factors, adverse outcomes, and hospital resource utilization. The Multicenter Study of Perioperative Ischemia Research Group. Ann Intern Med. 1998;128(3):194–203.PubMedCrossRef Mangano CM, Diamondstone LS, Ramsay JG, Aggarwal A, Herskowitz A, Mangano DT. Renal dysfunction after myocardial revascularization: risk factors, adverse outcomes, and hospital resource utilization. The Multicenter Study of Perioperative Ischemia Research Group. Ann Intern Med. 1998;128(3):194–203.PubMedCrossRef
3.
go back to reference Robert AM, Kramer RS, Dacey LJ, Charlesworth DC, Leavitt BJ, Helm RE, et al. Cardiac surgery-associated acute kidney injury: a comparison of two consensus criteria. Ann Thorac Surg. 2010;90(6):1939–43.PubMedCrossRef Robert AM, Kramer RS, Dacey LJ, Charlesworth DC, Leavitt BJ, Helm RE, et al. Cardiac surgery-associated acute kidney injury: a comparison of two consensus criteria. Ann Thorac Surg. 2010;90(6):1939–43.PubMedCrossRef
4.
go back to reference Brown JR, Cochran RP, Leavitt BJ, Dacey LJ, Ross CS, MacKenzie TA, et al. Multivariable prediction of renal insufficiency developing after cardiac surgery. Circulation. 2007;116(11 Suppl):I139–43.PubMed Brown JR, Cochran RP, Leavitt BJ, Dacey LJ, Ross CS, MacKenzie TA, et al. Multivariable prediction of renal insufficiency developing after cardiac surgery. Circulation. 2007;116(11 Suppl):I139–43.PubMed
5.
go back to reference Parolari A, Pesce LL, Pacini D, Mazzanti V, Salis S, Sciacovelli C, et al. Risk factors for perioperative acute kidney injury after adult cardiac surgery: role of perioperative management. Ann Thorac Surg. 2012;93(2):584–91.PubMedCrossRef Parolari A, Pesce LL, Pacini D, Mazzanti V, Salis S, Sciacovelli C, et al. Risk factors for perioperative acute kidney injury after adult cardiac surgery: role of perioperative management. Ann Thorac Surg. 2012;93(2):584–91.PubMedCrossRef
6.
go back to reference Biteker M, Dayan A, Tekkesin AI, Can MM, Tayci I, Ilhan E, et al. Incidence, risk factors, and outcomes of perioperative acute kidney injury in noncardiac and nonvascular surgery. Am J Surg. 2014;207(1):53–9.PubMedCrossRef Biteker M, Dayan A, Tekkesin AI, Can MM, Tayci I, Ilhan E, et al. Incidence, risk factors, and outcomes of perioperative acute kidney injury in noncardiac and nonvascular surgery. Am J Surg. 2014;207(1):53–9.PubMedCrossRef
8.
go back to reference Ortega-Loubon C, Fernandez-Molina M, Carrascal-Hinojal Y, Fulquet-Carreras E. Cardiac surgery-associated acute kidney injury. Ann Card Anaesth. 2016;19(4):687–98.PubMedPubMedCentralCrossRef Ortega-Loubon C, Fernandez-Molina M, Carrascal-Hinojal Y, Fulquet-Carreras E. Cardiac surgery-associated acute kidney injury. Ann Card Anaesth. 2016;19(4):687–98.PubMedPubMedCentralCrossRef
9.
go back to reference Mehta RH, Grab JD, O’Brien SM, Bridges CR, Gammie JS, Haan CK, et al. Bedside tool for predicting the risk of postoperative dialysis in patients undergoing cardiac surgery. Circulation. 2006;114(21):2208–16 (quiz).PubMedCrossRef Mehta RH, Grab JD, O’Brien SM, Bridges CR, Gammie JS, Haan CK, et al. Bedside tool for predicting the risk of postoperative dialysis in patients undergoing cardiac surgery. Circulation. 2006;114(21):2208–16 (quiz).PubMedCrossRef
10.
go back to reference Wijeysundera DN, Karkouti K, Dupuis JY, Rao V, Chan CT, Granton JT, et al. Derivation and validation of a simplified predictive index for renal replacement therapy after cardiac surgery. JAMA. 2007;297(16):1801–9.PubMedCrossRef Wijeysundera DN, Karkouti K, Dupuis JY, Rao V, Chan CT, Granton JT, et al. Derivation and validation of a simplified predictive index for renal replacement therapy after cardiac surgery. JAMA. 2007;297(16):1801–9.PubMedCrossRef
11.
go back to reference Thakar CV, Liangos O, Yared JP, Nelson DA, Hariachar S, Paganini EP. Predicting acute renal failure after cardiac surgery: validation and re-definition of a risk-stratification algorithm. Hemodial Int. 2003;7(2):143–7.PubMedCrossRef Thakar CV, Liangos O, Yared JP, Nelson DA, Hariachar S, Paganini EP. Predicting acute renal failure after cardiac surgery: validation and re-definition of a risk-stratification algorithm. Hemodial Int. 2003;7(2):143–7.PubMedCrossRef
13.
go back to reference Palomba H, de Castro I, Neto AL, Lage S, Yu L. Acute kidney injury prediction following elective cardiac surgery: AKICS Score. Kidney Int. 2007;72(5):624–31.PubMedCrossRef Palomba H, de Castro I, Neto AL, Lage S, Yu L. Acute kidney injury prediction following elective cardiac surgery: AKICS Score. Kidney Int. 2007;72(5):624–31.PubMedCrossRef
14.
go back to reference Lok CE, Austin PC, Wang H, Tu JV. Impact of renal insufficiency on short- and long-term outcomes after cardiac surgery. Am Heart J. 2004;148(3):430–8.PubMedCrossRef Lok CE, Austin PC, Wang H, Tu JV. Impact of renal insufficiency on short- and long-term outcomes after cardiac surgery. Am Heart J. 2004;148(3):430–8.PubMedCrossRef
15.
go back to reference Gurm HS, Kooiman J, LaLonde T, Grines C, Share D, Seth M. A random forest based risk model for reliable and accurate prediction of receipt of transfusion in patients undergoing percutaneous coronary intervention. PLoS ONE. 2014;9(5): e96385.PubMedPubMedCentralCrossRef Gurm HS, Kooiman J, LaLonde T, Grines C, Share D, Seth M. A random forest based risk model for reliable and accurate prediction of receipt of transfusion in patients undergoing percutaneous coronary intervention. PLoS ONE. 2014;9(5): e96385.PubMedPubMedCentralCrossRef
16.
go back to reference Ward MM, Pajevic S, Dreyfuss J, Malley JD. Short-term prediction of mortality in patients with systemic lupus erythematosus: classification of outcomes using random forests. Arthritis Rheum. 2006;55(1):74–80.PubMedCrossRef Ward MM, Pajevic S, Dreyfuss J, Malley JD. Short-term prediction of mortality in patients with systemic lupus erythematosus: classification of outcomes using random forests. Arthritis Rheum. 2006;55(1):74–80.PubMedCrossRef
17.
go back to reference Li J, Tran M, Siwabessy J. Selecting optimal random forest predictive models: a case study on predicting the spatial distribution of seabed hardness. PLoS ONE. 2016;11(2): e0149089.PubMedPubMedCentralCrossRef Li J, Tran M, Siwabessy J. Selecting optimal random forest predictive models: a case study on predicting the spatial distribution of seabed hardness. PLoS ONE. 2016;11(2): e0149089.PubMedPubMedCentralCrossRef
18.
go back to reference Sun LY, Chung AM, Farkouh ME, van Diepen S, Weinberger J, Bourke M, et al. Defining an intraoperative hypotension threshold in association with stroke in cardiac surgery. Anesthesiology. 2018;129(3):440–7.PubMedCrossRef Sun LY, Chung AM, Farkouh ME, van Diepen S, Weinberger J, Bourke M, et al. Defining an intraoperative hypotension threshold in association with stroke in cardiac surgery. Anesthesiology. 2018;129(3):440–7.PubMedCrossRef
19.
go back to reference Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int Suppl. 2012;2:1–138. Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int Suppl. 2012;2:1–138.
20.
go back to reference Birnie K, Verheyden V, Pagano D, Bhabra M, Tilling K, Sterne JA, et al. Predictive models for kidney disease: improving global outcomes (KDIGO) defined acute kidney injury in UK cardiac surgery. Crit Care. 2014;18(6):606.PubMedPubMedCentralCrossRef Birnie K, Verheyden V, Pagano D, Bhabra M, Tilling K, Sterne JA, et al. Predictive models for kidney disease: improving global outcomes (KDIGO) defined acute kidney injury in UK cardiac surgery. Crit Care. 2014;18(6):606.PubMedPubMedCentralCrossRef
21.
go back to reference Billings FT, Pretorius M, Schildcrout JS, Mercaldo ND, Byrne JG, Ikizler TA, et al. Obesity and oxidative stress predict AKI after cardiac surgery. J Am Soc Nephrol. 2012;23(7):1221–8.PubMedPubMedCentralCrossRef Billings FT, Pretorius M, Schildcrout JS, Mercaldo ND, Byrne JG, Ikizler TA, et al. Obesity and oxidative stress predict AKI after cardiac surgery. J Am Soc Nephrol. 2012;23(7):1221–8.PubMedPubMedCentralCrossRef
22.
go back to reference Ng SY, Sanagou M, Wolfe R, Cochrane A, Smith JA, Reid CM. Prediction of acute kidney injury within 30 days of cardiac surgery. J Thorac Cardiovasc Surg. 2014;147(6):1875–83.PubMedCrossRef Ng SY, Sanagou M, Wolfe R, Cochrane A, Smith JA, Reid CM. Prediction of acute kidney injury within 30 days of cardiac surgery. J Thorac Cardiovasc Surg. 2014;147(6):1875–83.PubMedCrossRef
23.
go back to reference Karkouti K, Grocott HP, Hall R, Jessen ME, Kruger C, Lerner AB, et al. Interrelationship of preoperative anemia, intraoperative anemia, and red blood cell transfusion as potentially modifiable risk factors for acute kidney injury in cardiac surgery: a historical multicentre cohort study. Can J Anaesth. 2015;62(4):377–84.PubMedCrossRef Karkouti K, Grocott HP, Hall R, Jessen ME, Kruger C, Lerner AB, et al. Interrelationship of preoperative anemia, intraoperative anemia, and red blood cell transfusion as potentially modifiable risk factors for acute kidney injury in cardiac surgery: a historical multicentre cohort study. Can J Anaesth. 2015;62(4):377–84.PubMedCrossRef
24.
go back to reference Dupuis JY, Wang F, Nathan H, Lam M, Grimes S, Bourke M. The cardiac anesthesia risk evaluation score: a clinically useful predictor of mortality and morbidity after cardiac surgery. Anesthesiology. 2001;94(2):194–204.PubMedCrossRef Dupuis JY, Wang F, Nathan H, Lam M, Grimes S, Bourke M. The cardiac anesthesia risk evaluation score: a clinically useful predictor of mortality and morbidity after cardiac surgery. Anesthesiology. 2001;94(2):194–204.PubMedCrossRef
25.
go back to reference Tran DT, Dupuis JY, Mesana T, Ruel M, Nathan HJ. Comparison of the EuroSCORE and Cardiac Anesthesia Risk Evaluation (CARE) score for risk-adjusted mortality analysis in cardiac surgery. Eur J Cardiothorac Surg. 2012;41(2):307–13.PubMedCrossRef Tran DT, Dupuis JY, Mesana T, Ruel M, Nathan HJ. Comparison of the EuroSCORE and Cardiac Anesthesia Risk Evaluation (CARE) score for risk-adjusted mortality analysis in cardiac surgery. Eur J Cardiothorac Surg. 2012;41(2):307–13.PubMedCrossRef
27.
go back to reference Legrand M, Pirracchio R, Rosa A, Petersen ML, Van der Laan M, Fabiani JN, et al. Incidence, risk factors and prediction of post-operative acute kidney injury following cardiac surgery for active infective endocarditis: an observational study. Crit Care. 2013;17(5):R220.PubMedPubMedCentralCrossRef Legrand M, Pirracchio R, Rosa A, Petersen ML, Van der Laan M, Fabiani JN, et al. Incidence, risk factors and prediction of post-operative acute kidney injury following cardiac surgery for active infective endocarditis: an observational study. Crit Care. 2013;17(5):R220.PubMedPubMedCentralCrossRef
28.
go back to reference Karkouti K, Wijeysundera DN, Yau TM, Callum JL, Cheng DC, Crowther M, et al. Acute kidney injury after cardiac surgery: focus on modifiable risk factors. Circulation. 2009;119(4):495–502.PubMedCrossRef Karkouti K, Wijeysundera DN, Yau TM, Callum JL, Cheng DC, Crowther M, et al. Acute kidney injury after cardiac surgery: focus on modifiable risk factors. Circulation. 2009;119(4):495–502.PubMedCrossRef
29.
go back to reference Bahar I, Akgul A, Ozatik MA, Vural KM, Demirbag AE, Boran M, et al. Acute renal failure following open heart surgery: risk factors and prognosis. Perfusion. 2005;20(6):317–22.PubMedCrossRef Bahar I, Akgul A, Ozatik MA, Vural KM, Demirbag AE, Boran M, et al. Acute renal failure following open heart surgery: risk factors and prognosis. Perfusion. 2005;20(6):317–22.PubMedCrossRef
30.
go back to reference Austin PC. Using the bootstrap to improve estimation and confidence intervals for regression coefficients selected using backwards variable elimination. Stat Med. 2008;27(17):3286–300.PubMedCrossRef Austin PC. Using the bootstrap to improve estimation and confidence intervals for regression coefficients selected using backwards variable elimination. Stat Med. 2008;27(17):3286–300.PubMedCrossRef
32.
go back to reference Liam A, Wiener M. Classification and regression by random forest. R News. 2002;2(3):315–26. Liam A, Wiener M. Classification and regression by random forest. R News. 2002;2(3):315–26.
33.
go back to reference Touw WG, Bayjanov JR, Overmars L, Backus L, Boekhorst J, Wels M, et al. Data mining in the life sciences with random forest: a walk in the park or lost in the jungle? Brief Bioinform. 2013;14(3):315–26.PubMedCrossRef Touw WG, Bayjanov JR, Overmars L, Backus L, Boekhorst J, Wels M, et al. Data mining in the life sciences with random forest: a walk in the park or lost in the jungle? Brief Bioinform. 2013;14(3):315–26.PubMedCrossRef
35.
go back to reference Doerken S, Avalos M, Lagarde E, Schumacher M. Penalized logistic regression with low prevalence exposures beyond high dimensional settings. PLoS ONE. 2019;14(5): e0217057.PubMedPubMedCentralCrossRef Doerken S, Avalos M, Lagarde E, Schumacher M. Penalized logistic regression with low prevalence exposures beyond high dimensional settings. PLoS ONE. 2019;14(5): e0217057.PubMedPubMedCentralCrossRef
36.
go back to reference Flom P, Cassell D. Stopping stepwise: Why stepwise and similar selection methods are bad, and what you should use. In: Proceedings of the Northeast SAS User Group (NESUG). 2007. Flom P, Cassell D. Stopping stepwise: Why stepwise and similar selection methods are bad, and what you should use. In: Proceedings of the Northeast SAS User Group (NESUG). 2007.
37.
go back to reference van Walraven C, Jackson TD, Daneman N. Derivation and validation of the surgical site infections risk model using health administrative data. Infect Control Hosp Epidemiol. 2016;37(4):455–65.PubMedCrossRef van Walraven C, Jackson TD, Daneman N. Derivation and validation of the surgical site infections risk model using health administrative data. Infect Control Hosp Epidemiol. 2016;37(4):455–65.PubMedCrossRef
38.
go back to reference Sullivan LM, Massaro JM, D’Agostino RB Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med. 2004;23(10):1631–60.PubMedCrossRef Sullivan LM, Massaro JM, D’Agostino RB Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med. 2004;23(10):1631–60.PubMedCrossRef
39.
go back to reference Streiner DL, Cairney J. What’s under the ROC? An introduction to receiver operating characteristics curves. Can J Psychiatry. 2007;52(2):121–8.PubMedCrossRef Streiner DL, Cairney J. What’s under the ROC? An introduction to receiver operating characteristics curves. Can J Psychiatry. 2007;52(2):121–8.PubMedCrossRef
40.
go back to reference Sainani KL. Multivariate regression: the pitfalls of automated variable selection. PM&R. 2013;5(9):791–4.CrossRef Sainani KL. Multivariate regression: the pitfalls of automated variable selection. PM&R. 2013;5(9):791–4.CrossRef
41.
go back to reference Machado MN, Nakazone MA, Maia LN. Prognostic value of acute kidney injury after cardiac surgery according to kidney disease: improving global outcomes definition and staging (KDIGO) criteria. PLoS ONE. 2014;9(5): e98028.PubMedPubMedCentralCrossRef Machado MN, Nakazone MA, Maia LN. Prognostic value of acute kidney injury after cardiac surgery according to kidney disease: improving global outcomes definition and staging (KDIGO) criteria. PLoS ONE. 2014;9(5): e98028.PubMedPubMedCentralCrossRef
42.
go back to reference Kang HC, Chung MY. Images in clinical medicine. Peripheral artery disease. N Engl J Med. 2007;357(18): e19.PubMedCrossRef Kang HC, Chung MY. Images in clinical medicine. Peripheral artery disease. N Engl J Med. 2007;357(18): e19.PubMedCrossRef
43.
go back to reference Cremer J, Martin M, Redl H, Bahrami S, Abraham C, Graeter T, et al. Systemic inflammatory response syndrome after cardiac operations. Ann Thorac Surg. 1996;61(6):1714–20.PubMedCrossRef Cremer J, Martin M, Redl H, Bahrami S, Abraham C, Graeter T, et al. Systemic inflammatory response syndrome after cardiac operations. Ann Thorac Surg. 1996;61(6):1714–20.PubMedCrossRef
44.
go back to reference Sgouralis I, Evans RG, Layton AT. Renal medullary and urinary oxygen tension during cardiopulmonary bypass in the rat. Math Med Biol. 2017;34(3):313–33.PubMed Sgouralis I, Evans RG, Layton AT. Renal medullary and urinary oxygen tension during cardiopulmonary bypass in the rat. Math Med Biol. 2017;34(3):313–33.PubMed
45.
go back to reference Maroco J, Silva D, Rodrigues A, Guerreiro M, Santana I, de Mendonca A. Data mining methods in the prediction of dementia: a real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forests. BMC Res Notes. 2011;4:299.PubMedPubMedCentralCrossRef Maroco J, Silva D, Rodrigues A, Guerreiro M, Santana I, de Mendonca A. Data mining methods in the prediction of dementia: a real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forests. BMC Res Notes. 2011;4:299.PubMedPubMedCentralCrossRef
46.
go back to reference Ozcift A. Enhanced cancer recognition system based on random forests feature elimination algorithm. J Med Syst. 2012;36(4):2577–85.PubMedCrossRef Ozcift A. Enhanced cancer recognition system based on random forests feature elimination algorithm. J Med Syst. 2012;36(4):2577–85.PubMedCrossRef
47.
go back to reference Jorge-Monjas P, Bustamante-Munguira J, Lorenzo M, Heredia-Rodriguez M, Fierro I, Gomez-Sanchez E, et al. Predicting cardiac surgery-associated acute kidney injury: the CRATE score. J Crit Care. 2016;31(1):130–8.PubMedCrossRef Jorge-Monjas P, Bustamante-Munguira J, Lorenzo M, Heredia-Rodriguez M, Fierro I, Gomez-Sanchez E, et al. Predicting cardiac surgery-associated acute kidney injury: the CRATE score. J Crit Care. 2016;31(1):130–8.PubMedCrossRef
48.
go back to reference Austin PC, Lee DS, D’Agostino RB, Fine JP. Developing points-based risk-scoring systems in the presence of competing risks. Stat Med. 2018;37(8):1405.PubMedCrossRef Austin PC, Lee DS, D’Agostino RB, Fine JP. Developing points-based risk-scoring systems in the presence of competing risks. Stat Med. 2018;37(8):1405.PubMedCrossRef
49.
50.
go back to reference Nigwekar SU, Kandula P, Hix JK, Thakar CV. Off-pump coronary artery bypass surgery and acute kidney injury: a meta-analysis of randomized and observational studies. Am J Kidney Dis. 2009;54(3):413–23.PubMedCrossRef Nigwekar SU, Kandula P, Hix JK, Thakar CV. Off-pump coronary artery bypass surgery and acute kidney injury: a meta-analysis of randomized and observational studies. Am J Kidney Dis. 2009;54(3):413–23.PubMedCrossRef
Metadata
Title
Prediction of acute kidney injury risk after cardiac surgery: using a hybrid machine learning algorithm
Authors
Yelena Petrosyan
Thierry G. Mesana
Louise Y. Sun
Publication date
01-12-2022
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2022
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-022-01859-w

Other articles of this Issue 1/2022

BMC Medical Informatics and Decision Making 1/2022 Go to the issue