Skip to main content
Top
Published in: BMC Medical Informatics and Decision Making 1/2023

Open Access 01-12-2023 | Research

Machine learning for the prediction of sepsis-related death: a systematic review and meta-analysis

Authors: Yan Zhang, Weiwei Xu, Ping Yang, An Zhang

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Login to get access

Abstract

Background and objectives

Sepsis is accompanied by a considerably high risk of mortality in the short term, despite the availability of recommended mortality risk assessment tools. However, these risk assessment tools seem to have limited predictive value. With the gradual integration of machine learning into clinical practice, some researchers have attempted to employ machine learning for early mortality risk prediction in sepsis patients. Nevertheless, there is a lack of comprehensive understanding regarding the construction of predictive variables using machine learning and the value of various machine learning methods. Thus, we carried out this systematic review and meta-analysis to explore the predictive value of machine learning for sepsis-related death at different time points.

Methods

PubMed, Embase, Cochrane, and Web of Science databases were searched until August 9th, 2022. The risk of bias in predictive models was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). We also performed subgroup analysis according to time of death and type of model and summarized current predictive variables used to construct models for sepsis death prediction.

Results

Fifty original studies were included, covering 104 models. The combined Concordance index (C-index), sensitivity, and specificity of machine learning models were 0.799, 0.81, and 0.80 in the training set, and 0.774, 0.71, and 0.68 in the validation set, respectively. Machine learning outperformed conventional clinical scoring tools and showed excellent C-index, sensitivity, and specificity in different subgroups. Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) are the preferred machine learning models because they showed more favorable accuracy with similar modeling variables. This study found that lactate was the most frequent predictor but was seriously ignored by current clinical scoring tools.

Conclusion

Machine learning methods demonstrate relatively favorable accuracy in predicting the mortality risk in sepsis patients. Given the limitations in accuracy and applicability of existing prediction scoring systems, there is an opportunity to explore updates based on existing machine learning approaches. Specifically, it is essential to develop or update more suitable mortality risk assessment tools based on the specific contexts of use, such as emergency departments, general wards, and intensive care units.
Appendix
Available only for authorised users
Literature
1.
go back to reference Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, et al. The third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–10.PubMedPubMedCentralCrossRef Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, et al. The third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–10.PubMedPubMedCentralCrossRef
2.
go back to reference Reinhart K, Daniels R, Kissoon N, Machado FR, Schachter RD, Finfer S. Recognizing sepsis as a global health priority - A WHO resolution. N Engl J Med. 2017;377(5):414–7.PubMedCrossRef Reinhart K, Daniels R, Kissoon N, Machado FR, Schachter RD, Finfer S. Recognizing sepsis as a global health priority - A WHO resolution. N Engl J Med. 2017;377(5):414–7.PubMedCrossRef
3.
go back to reference Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, Angus DC, Reinhart K. Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations. Am J Respir Crit Care Med. 2016;193(3):259–72.PubMedCrossRef Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, Angus DC, Reinhart K. Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations. Am J Respir Crit Care Med. 2016;193(3):259–72.PubMedCrossRef
4.
go back to reference Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, Colombara DV, Ikuta KS, Kissoon N, Finfer S, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the global burden of disease study. Lancet (London England). 2020;395(10219):200–11.PubMedCrossRef Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, Colombara DV, Ikuta KS, Kissoon N, Finfer S, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the global burden of disease study. Lancet (London England). 2020;395(10219):200–11.PubMedCrossRef
5.
go back to reference Vincent JL, Marshall JC, Namendys-Silva SA, François B, Martin-Loeches I, Lipman J, Reinhart K, Antonelli M, Pickkers P, Njimi H, et al. Assessment of the worldwide burden of critical Illness: the intensive care over nations (ICON) audit. The Lancet Respiratory Medicine. 2014;2(5):380–6.PubMedCrossRef Vincent JL, Marshall JC, Namendys-Silva SA, François B, Martin-Loeches I, Lipman J, Reinhart K, Antonelli M, Pickkers P, Njimi H, et al. Assessment of the worldwide burden of critical Illness: the intensive care over nations (ICON) audit. The Lancet Respiratory Medicine. 2014;2(5):380–6.PubMedCrossRef
6.
go back to reference Fleischmann-Struzek C, Mellhammar L, Rose N, Cassini A, Rudd KE, Schlattmann P, Allegranzi B, Reinhart K. Incidence and mortality of hospital- and ICU-treated sepsis: results from an updated and expanded systematic review and meta-analysis. Intensive Care Med. 2020;46(8):1552–62.PubMedPubMedCentralCrossRef Fleischmann-Struzek C, Mellhammar L, Rose N, Cassini A, Rudd KE, Schlattmann P, Allegranzi B, Reinhart K. Incidence and mortality of hospital- and ICU-treated sepsis: results from an updated and expanded systematic review and meta-analysis. Intensive Care Med. 2020;46(8):1552–62.PubMedPubMedCentralCrossRef
7.
go back to reference Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–63.PubMedCrossRef Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–63.PubMedCrossRef
8.
go back to reference Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.PubMedCrossRef Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.PubMedCrossRef
9.
go back to reference Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, Reinhart CK, Suter PM, Thijs LG. The SOFA (Sepsis-related Organ failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-related problems of the European society of intensive care medicine. Intensive Care Med. 1996;22(7):707–10.PubMedCrossRef Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, Reinhart CK, Suter PM, Thijs LG. The SOFA (Sepsis-related Organ failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-related problems of the European society of intensive care medicine. Intensive Care Med. 1996;22(7):707–10.PubMedCrossRef
10.
go back to reference Zygun DA, Laupland KB, Fick GH, Sandham JD, Doig CJ. Limited ability of SOFA and MOD scores to discriminate outcome: a prospective evaluation in 1,436 patients. Can J Anaesth = J Canadien D’anesthesie. 2005;52(3):302–8.CrossRef Zygun DA, Laupland KB, Fick GH, Sandham JD, Doig CJ. Limited ability of SOFA and MOD scores to discriminate outcome: a prospective evaluation in 1,436 patients. Can J Anaesth = J Canadien D’anesthesie. 2005;52(3):302–8.CrossRef
11.
go back to reference Khwannimit B, Bhurayanontachai R, Vattanavanit V. Validation of the sepsis severity score compared with updated severity scores in predicting hospital mortality in sepsis patients. Shock (Augusta Ga). 2017;47(6):720–5.PubMedCrossRef Khwannimit B, Bhurayanontachai R, Vattanavanit V. Validation of the sepsis severity score compared with updated severity scores in predicting hospital mortality in sepsis patients. Shock (Augusta Ga). 2017;47(6):720–5.PubMedCrossRef
12.
go back to reference Liu Z, Meng Z, Li Y, Zhao J, Wu S, Gou S, Wu H. Prognostic accuracy of the serum lactate level, the SOFA score and the qSOFA score for mortality among adults with Sepsis. Scand J Trauma Resusc Emerg Med. 2019;27(1):51.PubMedPubMedCentralCrossRef Liu Z, Meng Z, Li Y, Zhao J, Wu S, Gou S, Wu H. Prognostic accuracy of the serum lactate level, the SOFA score and the qSOFA score for mortality among adults with Sepsis. Scand J Trauma Resusc Emerg Med. 2019;27(1):51.PubMedPubMedCentralCrossRef
13.
go back to reference Fernando SM, Tran A, Taljaard M, Cheng W, Rochwerg B, Seely AJE, Perry JJ. Prognostic accuracy of the Quick Sequential Organ failure Assessment for Mortality in patients with suspected Infection: a systematic review and Meta-analysis. Ann Intern Med. 2018;168(4):266–75.PubMedCrossRef Fernando SM, Tran A, Taljaard M, Cheng W, Rochwerg B, Seely AJE, Perry JJ. Prognostic accuracy of the Quick Sequential Organ failure Assessment for Mortality in patients with suspected Infection: a systematic review and Meta-analysis. Ann Intern Med. 2018;168(4):266–75.PubMedCrossRef
14.
go back to reference Li R, Chen Y, Ritchie MD, Moore JH. Electronic health records and polygenic risk scores for predicting Disease risk. Nat Rev Genet. 2020;21(8):493–502.PubMedCrossRef Li R, Chen Y, Ritchie MD, Moore JH. Electronic health records and polygenic risk scores for predicting Disease risk. Nat Rev Genet. 2020;21(8):493–502.PubMedCrossRef
15.
go back to reference Leong KT, Wong LY, Aung KC, Macdonald M, Cao Y, Lee S, Chow WL, Doddamani S, Richards AM. Risk stratification model for 30-Day Heart Failure readmission in a multiethnic South East Asian Community. Am J Cardiol. 2017;119(9):1428–32.PubMedCrossRef Leong KT, Wong LY, Aung KC, Macdonald M, Cao Y, Lee S, Chow WL, Doddamani S, Richards AM. Risk stratification model for 30-Day Heart Failure readmission in a multiethnic South East Asian Community. Am J Cardiol. 2017;119(9):1428–32.PubMedCrossRef
16.
go back to reference Heo J, Yoon JG, Park H, Kim YD, Nam HS, Heo JH. Machine learning-based model for prediction of outcomes in Acute Stroke. Stroke. 2019;50(5):1263–5.PubMedCrossRef Heo J, Yoon JG, Park H, Kim YD, Nam HS, Heo JH. Machine learning-based model for prediction of outcomes in Acute Stroke. Stroke. 2019;50(5):1263–5.PubMedCrossRef
18.
go back to reference Gultepe E, Green JP, Nguyen H, Adams J, Albertson T, Tagkopoulos I. From vital signs to clinical outcomes for patients with sepsis: a machine learning basis for a clinical decision support system. J Am Med Inf Association: JAMIA. 2014;21(2):315–25.CrossRef Gultepe E, Green JP, Nguyen H, Adams J, Albertson T, Tagkopoulos I. From vital signs to clinical outcomes for patients with sepsis: a machine learning basis for a clinical decision support system. J Am Med Inf Association: JAMIA. 2014;21(2):315–25.CrossRef
19.
go back to reference Pirracchio R, Petersen ML, Carone M, Rigon MR, Chevret S, van der Laan MJ. Mortality prediction in intensive care units with the Super ICU Learner Algorithm (SICULA): a population-based study. The Lancet Respiratory Medicine. 2015;3(1):42–52.PubMedCrossRef Pirracchio R, Petersen ML, Carone M, Rigon MR, Chevret S, van der Laan MJ. Mortality prediction in intensive care units with the Super ICU Learner Algorithm (SICULA): a population-based study. The Lancet Respiratory Medicine. 2015;3(1):42–52.PubMedCrossRef
20.
21.
go back to reference Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, Liu PJ, Liu X, Marcus J, Sun M, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med. 2018;1:18.PubMedPubMedCentralCrossRef Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, Liu PJ, Liu X, Marcus J, Sun M, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med. 2018;1:18.PubMedPubMedCentralCrossRef
22.
go back to reference Taylor RA, Pare JR, Venkatesh AK, Mowafi H, Melnick ER, Fleischman W, Hall MK. Prediction of In-hospital mortality in Emergency Department patients with Sepsis: a local Big Data-Driven, Machine Learning Approach. Acad Emerg Medicine: Official J Soc Acad Emerg Med. 2016;23(3):269–78.CrossRef Taylor RA, Pare JR, Venkatesh AK, Mowafi H, Melnick ER, Fleischman W, Hall MK. Prediction of In-hospital mortality in Emergency Department patients with Sepsis: a local Big Data-Driven, Machine Learning Approach. Acad Emerg Medicine: Official J Soc Acad Emerg Med. 2016;23(3):269–78.CrossRef
23.
go back to reference Fleuren LM, Klausch TLT, Zwager CL, Schoonmade LJ, Guo T, Roggeveen LF, Swart EL, Girbes ARJ, Thoral P, Ercole A, et al. Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med. 2020;46(3):383–400.PubMedPubMedCentralCrossRef Fleuren LM, Klausch TLT, Zwager CL, Schoonmade LJ, Guo T, Roggeveen LF, Swart EL, Girbes ARJ, Thoral P, Ercole A, et al. Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med. 2020;46(3):383–400.PubMedPubMedCentralCrossRef
24.
go back to reference Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ (Clinical Research ed). 2021;372:n71.PubMed Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ (Clinical Research ed). 2021;372:n71.PubMed
25.
go back to reference Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, Topol EJ, Ioannidis JPA, Collins GS, Maruthappu M. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ (Clinical Research ed). 2020;368:m689.PubMed Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, Topol EJ, Ioannidis JPA, Collins GS, Maruthappu M. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ (Clinical Research ed). 2020;368:m689.PubMed
26.
go back to reference Debray TP, Damen JA, Riley RD, Snell K, Reitsma JB, Hooft L, Collins GS, Moons KG. A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes. Stat Methods Med Res. 2019;28(9):2768–86.PubMedCrossRef Debray TP, Damen JA, Riley RD, Snell K, Reitsma JB, Hooft L, Collins GS, Moons KG. A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes. Stat Methods Med Res. 2019;28(9):2768–86.PubMedCrossRef
27.
go back to reference Zhi D, Zhang M, Lin J, Liu P, Wang Y, Duan M. Establishment and validation of the predictive model for the in-hospital death in patients with sepsis. Am J Infect Control. 2021;49(12):1515–21.PubMedCrossRef Zhi D, Zhang M, Lin J, Liu P, Wang Y, Duan M. Establishment and validation of the predictive model for the in-hospital death in patients with sepsis. Am J Infect Control. 2021;49(12):1515–21.PubMedCrossRef
28.
go back to reference Zhao C, Wei Y, Chen D, Jin J, Chen H. Prognostic value of an inflammatory biomarker-based clinical algorithm in septic patients in the emergency department: an observational study. Int Immunopharmacol. 2020;80:106145.PubMedCrossRef Zhao C, Wei Y, Chen D, Jin J, Chen H. Prognostic value of an inflammatory biomarker-based clinical algorithm in septic patients in the emergency department: an observational study. Int Immunopharmacol. 2020;80:106145.PubMedCrossRef
29.
go back to reference Zhang L, Huang T, Xu F, Li S, Zheng S, Lyu J, Yin H. Prediction of prognosis in elderly patients with sepsis based on machine learning (random survival forest). BMC Emerg Med. 2022;22(1):26.PubMedPubMedCentralCrossRef Zhang L, Huang T, Xu F, Li S, Zheng S, Lyu J, Yin H. Prediction of prognosis in elderly patients with sepsis based on machine learning (random survival forest). BMC Emerg Med. 2022;22(1):26.PubMedPubMedCentralCrossRef
30.
go back to reference Zhang K, Zhang S, Cui W, Hong Y, Zhang G, Zhang Z. Development and validation of a Sepsis mortality risk score for Sepsis-3 patients in Intensive Care Unit. Front Med. 2020;7:609769.CrossRef Zhang K, Zhang S, Cui W, Hong Y, Zhang G, Zhang Z. Development and validation of a Sepsis mortality risk score for Sepsis-3 patients in Intensive Care Unit. Front Med. 2020;7:609769.CrossRef
31.
go back to reference Zeng Z, Yao S, Zheng J, Gong X. Development and validation of a novel blending machine learning model for hospital mortality prediction in ICU patients with Sepsis. BioData Min. 2021;14(1):40.PubMedPubMedCentralCrossRef Zeng Z, Yao S, Zheng J, Gong X. Development and validation of a novel blending machine learning model for hospital mortality prediction in ICU patients with Sepsis. BioData Min. 2021;14(1):40.PubMedPubMedCentralCrossRef
32.
go back to reference Zeng Q, He L, Zhang N, Lin Q, Zhong L, Song J. Prediction of 90-Day Mortality among Sepsis Patients Based on a Nomogram Integrating Diverse Clinical Indices. BioMed research international 2021, 2021:1023513. Zeng Q, He L, Zhang N, Lin Q, Zhong L, Song J. Prediction of 90-Day Mortality among Sepsis Patients Based on a Nomogram Integrating Diverse Clinical Indices. BioMed research international 2021, 2021:1023513.
33.
go back to reference Yao RQ, Jin X, Wang GW, Yu Y, Wu GS, Zhu YB, Li L, Li YX, Zhao PY, Zhu SY, et al. A machine learning-based prediction of hospital mortality in patients with postoperative Sepsis. Front Med. 2020;7:445.CrossRef Yao RQ, Jin X, Wang GW, Yu Y, Wu GS, Zhu YB, Li L, Li YX, Zhao PY, Zhu SY, et al. A machine learning-based prediction of hospital mortality in patients with postoperative Sepsis. Front Med. 2020;7:445.CrossRef
34.
go back to reference Wong HR, Lindsell CJ, Pettilä V, Meyer NJ, Thair SA, Karlsson S, Russell JA, Fjell CD, Boyd JH, Ruokonen E, et al. A multibiomarker-based outcome risk stratification model for adult septic shock*. Crit Care Med. 2014;42(4):781–9.PubMedPubMedCentralCrossRef Wong HR, Lindsell CJ, Pettilä V, Meyer NJ, Thair SA, Karlsson S, Russell JA, Fjell CD, Boyd JH, Ruokonen E, et al. A multibiomarker-based outcome risk stratification model for adult septic shock*. Crit Care Med. 2014;42(4):781–9.PubMedPubMedCentralCrossRef
35.
go back to reference Wernly B, Mamandipoor B, Baldia P, Jung C, Osmani V. Machine learning predicts mortality in septic patients using only routinely available ABG variables: a multi-centre evaluation. Int J Med Informatics. 2021;145:104312.CrossRef Wernly B, Mamandipoor B, Baldia P, Jung C, Osmani V. Machine learning predicts mortality in septic patients using only routinely available ABG variables: a multi-centre evaluation. Int J Med Informatics. 2021;145:104312.CrossRef
36.
go back to reference Wang H, Li Y, Naidech A, Luo Y. Comparison between machine learning methods for mortality prediction for sepsis patients with different social determinants. BMC Med Inf Decis Mak. 2022;22(Suppl 2):156.CrossRef Wang H, Li Y, Naidech A, Luo Y. Comparison between machine learning methods for mortality prediction for sepsis patients with different social determinants. BMC Med Inf Decis Mak. 2022;22(Suppl 2):156.CrossRef
37.
go back to reference Wang B, Chen J. Establishment and validation of a predictive model for mortality within 30 days in patients with sepsis-induced blood pressure drop: a retrospective analysis. PLoS ONE. 2021;16(5):e0252009.PubMedPubMedCentralCrossRef Wang B, Chen J. Establishment and validation of a predictive model for mortality within 30 days in patients with sepsis-induced blood pressure drop: a retrospective analysis. PLoS ONE. 2021;16(5):e0252009.PubMedPubMedCentralCrossRef
38.
go back to reference van Doorn W, Stassen PM, Borggreve HF, Schalkwijk MJ, Stoffers J, Bekers O, Meex SJR. A comparison of machine learning models versus clinical evaluation for mortality prediction in patients with sepsis. PLoS ONE. 2021;16(1):e0245157.PubMedPubMedCentralCrossRef van Doorn W, Stassen PM, Borggreve HF, Schalkwijk MJ, Stoffers J, Bekers O, Meex SJR. A comparison of machine learning models versus clinical evaluation for mortality prediction in patients with sepsis. PLoS ONE. 2021;16(1):e0245157.PubMedPubMedCentralCrossRef
39.
go back to reference Vallabhajosyula S, Jentzer JC, Kotecha AA, Murphree DH Jr., Barreto EF, Khanna AK, Iyer VN. Development and performance of a novel vasopressor-driven mortality prediction model in septic shock. Ann Intensiv Care. 2018;8(1):112.CrossRef Vallabhajosyula S, Jentzer JC, Kotecha AA, Murphree DH Jr., Barreto EF, Khanna AK, Iyer VN. Development and performance of a novel vasopressor-driven mortality prediction model in septic shock. Ann Intensiv Care. 2018;8(1):112.CrossRef
40.
go back to reference Tsoukalas A, Albertson T, Tagkopoulos I. From data to optimal decision making: a data-driven, probabilistic machine learning approach to decision support for patients with sepsis. JMIR Med Inf. 2015;3(1):e11.CrossRef Tsoukalas A, Albertson T, Tagkopoulos I. From data to optimal decision making: a data-driven, probabilistic machine learning approach to decision support for patients with sepsis. JMIR Med Inf. 2015;3(1):e11.CrossRef
41.
go back to reference Taneja I, Damhorst GL, Lopez-Espina C, Zhao SD, Zhu R, Khan S, White K, Kumar J, Vincent A, Yeh L, et al. Diagnostic and prognostic capabilities of a biomarker and EMR-based machine learning algorithm for sepsis. Clin Transl Sci. 2021;14(4):1578–89.PubMedPubMedCentralCrossRef Taneja I, Damhorst GL, Lopez-Espina C, Zhao SD, Zhu R, Khan S, White K, Kumar J, Vincent A, Yeh L, et al. Diagnostic and prognostic capabilities of a biomarker and EMR-based machine learning algorithm for sepsis. Clin Transl Sci. 2021;14(4):1578–89.PubMedPubMedCentralCrossRef
42.
go back to reference Su L, Xu Z, Chang F, Ma Y, Liu S, Jiang H, Wang H, Li D, Chen H, Zhou X, et al. Early Prediction of Mortality, Severity, and length of stay in the Intensive Care Unit of Sepsis patients based on Sepsis 3.0 by machine learning models. Front Med. 2021;8:664966.CrossRef Su L, Xu Z, Chang F, Ma Y, Liu S, Jiang H, Wang H, Li D, Chen H, Zhou X, et al. Early Prediction of Mortality, Severity, and length of stay in the Intensive Care Unit of Sepsis patients based on Sepsis 3.0 by machine learning models. Front Med. 2021;8:664966.CrossRef
43.
go back to reference Speiser JL, Karvellas CJ, Shumilak G, Sligl WI, Mirzanejad Y, Gurka D, Kumar A, Kumar A. Predicting in-hospital mortality in pneumonia-associated septic shock patients using a classification and regression tree: a nested cohort study. J Intensive care. 2018;6:66.PubMedPubMedCentralCrossRef Speiser JL, Karvellas CJ, Shumilak G, Sligl WI, Mirzanejad Y, Gurka D, Kumar A, Kumar A. Predicting in-hospital mortality in pneumonia-associated septic shock patients using a classification and regression tree: a nested cohort study. J Intensive care. 2018;6:66.PubMedPubMedCentralCrossRef
44.
go back to reference Samsudin MI, Liu N, Prabhakar SM, Chong SL, Kit Lye W, Koh ZX, Guo D, Rajesh R, Ho AFW, Ong MEH. A novel heart rate variability based risk prediction model for septic patients presenting to the emergency department. Medicine. 2018;97(23):e10866.PubMedPubMedCentralCrossRef Samsudin MI, Liu N, Prabhakar SM, Chong SL, Kit Lye W, Koh ZX, Guo D, Rajesh R, Ho AFW, Ong MEH. A novel heart rate variability based risk prediction model for septic patients presenting to the emergency department. Medicine. 2018;97(23):e10866.PubMedPubMedCentralCrossRef
45.
go back to reference Rodríguez A, Mendoza D, Ascuntar J, Jaimes F. Supervised classification techniques for prediction of mortality in adult patients with sepsis. Am J Emerg Med. 2021;45:392–7.PubMedCrossRef Rodríguez A, Mendoza D, Ascuntar J, Jaimes F. Supervised classification techniques for prediction of mortality in adult patients with sepsis. Am J Emerg Med. 2021;45:392–7.PubMedCrossRef
46.
go back to reference Ren Y, Zhang L, Xu F, Han D, Zheng S, Zhang F, Li L, Wang Z, Lyu J, Yin H. Risk factor analysis and nomogram for predicting in-hospital mortality in ICU patients with sepsis and lung Infection. BMC Pulm Med. 2022;22(1):17.PubMedPubMedCentralCrossRef Ren Y, Zhang L, Xu F, Han D, Zheng S, Zhang F, Li L, Wang Z, Lyu J, Yin H. Risk factor analysis and nomogram for predicting in-hospital mortality in ICU patients with sepsis and lung Infection. BMC Pulm Med. 2022;22(1):17.PubMedPubMedCentralCrossRef
47.
go back to reference Prabhakar SM, Tagami T, Liu N, Samsudin MI, Ng JCJ, Koh ZX, Ong MEH. Combining quick sequential organ failure assessment score with heart rate variability may improve predictive ability for mortality in septic patients at the emergency department. PLoS ONE. 2019;14(3):e0213445.PubMedPubMedCentralCrossRef Prabhakar SM, Tagami T, Liu N, Samsudin MI, Ng JCJ, Koh ZX, Ong MEH. Combining quick sequential organ failure assessment score with heart rate variability may improve predictive ability for mortality in septic patients at the emergency department. PLoS ONE. 2019;14(3):e0213445.PubMedPubMedCentralCrossRef
48.
go back to reference Perng JW, Kao IH, Kung CT, Hung SC, Lai YH, Su CM. Mortality prediction of septic patients in the Emergency Department based on machine learning. J Clin Med 2019, 8(11). Perng JW, Kao IH, Kung CT, Hung SC, Lai YH, Su CM. Mortality prediction of septic patients in the Emergency Department based on machine learning. J Clin Med 2019, 8(11).
49.
go back to reference Park JY, Hsu TC, Hu JR, Chen CY, Hsu WT, Lee M, Ho J, Lee CC. Predicting Sepsis Mortality in a Population-Based National Database: Machine Learning Approach. J Med Internet Res. 2022;24(4):e29982.PubMedPubMedCentralCrossRef Park JY, Hsu TC, Hu JR, Chen CY, Hsu WT, Lee M, Ho J, Lee CC. Predicting Sepsis Mortality in a Population-Based National Database: Machine Learning Approach. J Med Internet Res. 2022;24(4):e29982.PubMedPubMedCentralCrossRef
50.
go back to reference Liu N, Chee ML, Foo MZQ, Pong JZ, Guo D, Koh ZX, Ho AFW, Niu C, Chong SL, Ong MEH. Heart rate n-variability (HRnV) measures for prediction of mortality in sepsis patients presenting at the emergency department. PLoS ONE. 2021;16(8):e0249868.PubMedPubMedCentralCrossRef Liu N, Chee ML, Foo MZQ, Pong JZ, Guo D, Koh ZX, Ho AFW, Niu C, Chong SL, Ong MEH. Heart rate n-variability (HRnV) measures for prediction of mortality in sepsis patients presenting at the emergency department. PLoS ONE. 2021;16(8):e0249868.PubMedPubMedCentralCrossRef
51.
go back to reference Liu H, Zhang L, Xu F, Li S, Wang Z, Han D, Zhang F, Lyu J, Yin H. Establishment of a prognostic model for patients with sepsis based on SOFA: a retrospective cohort study. J Int Med Res. 2021;49(9):3000605211044892.PubMedCrossRef Liu H, Zhang L, Xu F, Li S, Wang Z, Han D, Zhang F, Lyu J, Yin H. Establishment of a prognostic model for patients with sepsis based on SOFA: a retrospective cohort study. J Int Med Res. 2021;49(9):3000605211044892.PubMedCrossRef
52.
go back to reference Li K, Shi Q, Liu S, Xie Y, Liu J. Predicting in-hospital mortality in ICU patients with sepsis using gradient boosting decision tree. Medicine. 2021;100(19):e25813.PubMedPubMedCentralCrossRef Li K, Shi Q, Liu S, Xie Y, Liu J. Predicting in-hospital mortality in ICU patients with sepsis using gradient boosting decision tree. Medicine. 2021;100(19):e25813.PubMedPubMedCentralCrossRef
53.
go back to reference Lagu T, Lindenauer PK, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS, Higgins TL. Development and validation of a model that uses enhanced administrative data to predict mortality in patients with sepsis. Crit Care Med. 2011;39(11):2425–30.PubMedCrossRef Lagu T, Lindenauer PK, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS, Higgins TL. Development and validation of a model that uses enhanced administrative data to predict mortality in patients with sepsis. Crit Care Med. 2011;39(11):2425–30.PubMedCrossRef
54.
go back to reference Kong G, Lin K, Hu Y. Using machine learning methods to predict in-hospital mortality of sepsis patients in the ICU. BMC Med Inf Decis Mak. 2020;20(1):251.CrossRef Kong G, Lin K, Hu Y. Using machine learning methods to predict in-hospital mortality of sepsis patients in the ICU. BMC Med Inf Decis Mak. 2020;20(1):251.CrossRef
55.
go back to reference Karlsson A, Stassen W, Loutfi A, Wallgren U, Larsson E, Kurland L. Predicting mortality among septic patients presenting to the emergency department-a cross sectional analysis using machine learning. BMC Emerg Med. 2021;21(1):84.PubMedPubMedCentralCrossRef Karlsson A, Stassen W, Loutfi A, Wallgren U, Larsson E, Kurland L. Predicting mortality among septic patients presenting to the emergency department-a cross sectional analysis using machine learning. BMC Emerg Med. 2021;21(1):84.PubMedPubMedCentralCrossRef
56.
go back to reference Jaimes F, Farbiarz J, Alvarez D, Martínez C. Comparison between logistic regression and neural networks to predict death in patients with suspected sepsis in the emergency room. Crit Care (London England). 2005;9(2):R150–156.CrossRef Jaimes F, Farbiarz J, Alvarez D, Martínez C. Comparison between logistic regression and neural networks to predict death in patients with suspected sepsis in the emergency room. Crit Care (London England). 2005;9(2):R150–156.CrossRef
57.
go back to reference Hu C, Li L, Huang W, Wu T, Xu Q, Liu J, Hu B. Interpretable Machine Learning for early prediction of prognosis in Sepsis: A Discovery and Validation Study. Infect Dis Therapy. 2022;11(3):1117–32.CrossRef Hu C, Li L, Huang W, Wu T, Xu Q, Liu J, Hu B. Interpretable Machine Learning for early prediction of prognosis in Sepsis: A Discovery and Validation Study. Infect Dis Therapy. 2022;11(3):1117–32.CrossRef
58.
go back to reference Hsu JF, Chang YF, Cheng HJ, Yang C, Lin CY, Chu SM, Huang HR, Chiang MC, Wang HC, Tsai MH. Machine learning approaches to Predict In-Hospital mortality among neonates with clinically suspected Sepsis in the neonatal Intensive Care Unit. J Personalized Med 2021, 11(8). Hsu JF, Chang YF, Cheng HJ, Yang C, Lin CY, Chu SM, Huang HR, Chiang MC, Wang HC, Tsai MH. Machine learning approaches to Predict In-Hospital mortality among neonates with clinically suspected Sepsis in the neonatal Intensive Care Unit. J Personalized Med 2021, 11(8).
59.
go back to reference Hou N, Li M, He L, Xie B, Wang L, Zhang R, Yu Y, Sun X, Pan Z, Wang K. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost. J Translational Med. 2020;18(1):462.CrossRef Hou N, Li M, He L, Xie B, Wang L, Zhang R, Yu Y, Sun X, Pan Z, Wang K. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost. J Translational Med. 2020;18(1):462.CrossRef
60.
go back to reference Hargovan S, Gunnarsson R, Carter A, De Costa A, Brooks J, Groch T, Sivalingam S. The 4-Hour Cairns Sepsis Model: a novel approach to predicting sepsis mortality at intensive care unit admission. Australian Crit care: Official J Confederation Australian Crit Care Nurses. 2021;34(6):552–60.CrossRef Hargovan S, Gunnarsson R, Carter A, De Costa A, Brooks J, Groch T, Sivalingam S. The 4-Hour Cairns Sepsis Model: a novel approach to predicting sepsis mortality at intensive care unit admission. Australian Crit care: Official J Confederation Australian Crit Care Nurses. 2021;34(6):552–60.CrossRef
61.
go back to reference Ghiasi S, Zhu T, Lu P, Hagenah J, Khanh PNQ, Hao NV, Vital C, Thwaites L, Clifton DA. Sepsis mortality prediction using Wearable Monitoring in Low-Middle Income Countries. Sensors 2022, 22(10). Ghiasi S, Zhu T, Lu P, Hagenah J, Khanh PNQ, Hao NV, Vital C, Thwaites L, Clifton DA. Sepsis mortality prediction using Wearable Monitoring in Low-Middle Income Countries. Sensors 2022, 22(10).
62.
go back to reference García-Gallo JE, Fonseca-Ruiz NJ, Celi LA, Duitama-Muñoz JF. A machine learning-based model for 1-year mortality prediction in patients admitted to an intensive care unit with a diagnosis of sepsis. Med Intensiva. 2020;44(3):160–70.PubMedCrossRef García-Gallo JE, Fonseca-Ruiz NJ, Celi LA, Duitama-Muñoz JF. A machine learning-based model for 1-year mortality prediction in patients admitted to an intensive care unit with a diagnosis of sepsis. Med Intensiva. 2020;44(3):160–70.PubMedCrossRef
63.
go back to reference Ford DW, Goodwin AJ, Simpson AN, Johnson E, Nadig N, Simpson KN. A severe Sepsis mortality prediction model and score for Use with Administrative Data. Crit Care Med. 2016;44(2):319–27.PubMedPubMedCentralCrossRef Ford DW, Goodwin AJ, Simpson AN, Johnson E, Nadig N, Simpson KN. A severe Sepsis mortality prediction model and score for Use with Administrative Data. Crit Care Med. 2016;44(2):319–27.PubMedPubMedCentralCrossRef
64.
go back to reference Chen M, Lu X, Hu L, Liu P, Zhao W, Yan H, Tang L, Zhu Y, Xiao Z, Chen L, et al. Development and validation of a mortality risk model for pediatric sepsis. Medicine. 2017;96(20):e6923.PubMedPubMedCentralCrossRef Chen M, Lu X, Hu L, Liu P, Zhao W, Yan H, Tang L, Zhu Y, Xiao Z, Chen L, et al. Development and validation of a mortality risk model for pediatric sepsis. Medicine. 2017;96(20):e6923.PubMedPubMedCentralCrossRef
65.
go back to reference Chao HY, Wu CC, Singh A, Shedd A, Wolfshohl J, Chou EH, Huang YC, Chen KF. Using machine learning to develop and validate an In-Hospital mortality prediction model for patients with suspected Sepsis. Biomedicines 2022, 10(4). Chao HY, Wu CC, Singh A, Shedd A, Wolfshohl J, Chou EH, Huang YC, Chen KF. Using machine learning to develop and validate an In-Hospital mortality prediction model for patients with suspected Sepsis. Biomedicines 2022, 10(4).
66.
go back to reference Phillips GS, Osborn TM, Terry KM, Gesten F, Levy MM, Lemeshow S. The New York Sepsis Severity score: development of a risk-adjusted severity model for Sepsis. Crit Care Med. 2018;46(5):674–83.PubMedCrossRef Phillips GS, Osborn TM, Terry KM, Gesten F, Levy MM, Lemeshow S. The New York Sepsis Severity score: development of a risk-adjusted severity model for Sepsis. Crit Care Med. 2018;46(5):674–83.PubMedCrossRef
67.
go back to reference Ribas Ripoll VJ, Vellido A, Romero E, Ruiz-Rodríguez JC. Sepsis mortality prediction with the quotient basis Kernel. Artif Intell Med. 2014;61(1):45–52.PubMedCrossRef Ribas Ripoll VJ, Vellido A, Romero E, Ruiz-Rodríguez JC. Sepsis mortality prediction with the quotient basis Kernel. Artif Intell Med. 2014;61(1):45–52.PubMedCrossRef
68.
go back to reference Guo X, Shuai XY, Cai TT, Wu ZY, Wu DW, Ding SF. The thrombodynamic ratio as a predictor of 28-day mortality in sepsis patients. Clin Chim Acta. 2022;531:399–405.PubMedCrossRef Guo X, Shuai XY, Cai TT, Wu ZY, Wu DW, Ding SF. The thrombodynamic ratio as a predictor of 28-day mortality in sepsis patients. Clin Chim Acta. 2022;531:399–405.PubMedCrossRef
69.
go back to reference Gong M, Liu J, Li C, Guo W, Wang R, Chen Z. Early warning model for death of sepsis via length insensitive temporal convolutional network. Med Biol Eng Comput. 2022;60(3):875–85.PubMedCrossRef Gong M, Liu J, Li C, Guo W, Wang R, Chen Z. Early warning model for death of sepsis via length insensitive temporal convolutional network. Med Biol Eng Comput. 2022;60(3):875–85.PubMedCrossRef
70.
go back to reference García-Gallo JE, Fonseca-Ruiz NJ, Duitama-Muñoz JF. Development of a Model that uses Data obtained in the admission to Predict one-year mortality in patients with Sepsis in the Intensive Care Unit. Int J Pharma Med Biol Sci 2019. García-Gallo JE, Fonseca-Ruiz NJ, Duitama-Muñoz JF. Development of a Model that uses Data obtained in the admission to Predict one-year mortality in patients with Sepsis in the Intensive Care Unit. Int J Pharma Med Biol Sci 2019.
71.
go back to reference Ding X, Tong R, Song H, Sun G, Wang D, Liang H, Sun J, Cui Y, Zhang X, Liu S, et al. Identification of metabolomics-based prognostic prediction models for ICU septic patients. Int Immunopharmacol. 2022;108:108841.PubMedCrossRef Ding X, Tong R, Song H, Sun G, Wang D, Liang H, Sun J, Cui Y, Zhang X, Liu S, et al. Identification of metabolomics-based prognostic prediction models for ICU septic patients. Int Immunopharmacol. 2022;108:108841.PubMedCrossRef
72.
go back to reference Wang W, Jie Y, Zhou J. Values of serum PCT, suPAR combined with severity scores for evaluating prognosis of septic shock patients. Revista Romana De Medicina De Laborator. 2021;29(4):395–402.CrossRef Wang W, Jie Y, Zhou J. Values of serum PCT, suPAR combined with severity scores for evaluating prognosis of septic shock patients. Revista Romana De Medicina De Laborator. 2021;29(4):395–402.CrossRef
73.
go back to reference Wang L, Tang C, He S, Chen Y, Xie C. Combined suPAR and qSOFA for the prediction of 28-day mortality in sepsis patients. Signa Vitae 2022, 18(3). Wang L, Tang C, He S, Chen Y, Xie C. Combined suPAR and qSOFA for the prediction of 28-day mortality in sepsis patients. Signa Vitae 2022, 18(3).
74.
go back to reference Selcuk M, Koc O, Kestel AS. The prediction power of machine learning on estimating the sepsis mortality in the intensive care unit. Inf Med Unlocked. 2022;28:100861.CrossRef Selcuk M, Koc O, Kestel AS. The prediction power of machine learning on estimating the sepsis mortality in the intensive care unit. Inf Med Unlocked. 2022;28:100861.CrossRef
75.
go back to reference Ribas VJ, López JC, Ruiz-Sanmartin A, Ruiz-Rodríguez JC, Rello J, Wojdel A, Vellido A. Severe sepsis mortality prediction with relevance vector machines. Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual International Conference 2011, 2011:100–103. Ribas VJ, López JC, Ruiz-Sanmartin A, Ruiz-Rodríguez JC, Rello J, Wojdel A, Vellido A. Severe sepsis mortality prediction with relevance vector machines. Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual International Conference 2011, 2011:100–103.
76.
go back to reference Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo R, Pilcher DV. Prognostic accuracy of the SOFA score, SIRS Criteria, and qSOFA score for In-Hospital mortality among adults with suspected Infection admitted to the Intensive Care Unit. JAMA. 2017;317(3):290–300.PubMedCrossRef Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo R, Pilcher DV. Prognostic accuracy of the SOFA score, SIRS Criteria, and qSOFA score for In-Hospital mortality among adults with suspected Infection admitted to the Intensive Care Unit. JAMA. 2017;317(3):290–300.PubMedCrossRef
77.
go back to reference Houwink AP, Rijkenberg S, Bosman RJ, van der Voort PH. The association between lactate, mean arterial pressure, central venous oxygen saturation and peripheral temperature and mortality in severe sepsis: a retrospective cohort analysis. Crit Care (London England). 2016;20:56.CrossRef Houwink AP, Rijkenberg S, Bosman RJ, van der Voort PH. The association between lactate, mean arterial pressure, central venous oxygen saturation and peripheral temperature and mortality in severe sepsis: a retrospective cohort analysis. Crit Care (London England). 2016;20:56.CrossRef
78.
go back to reference Ryoo SM, Lee J, Lee YS, Lee JH, Lim KS, Huh JW, Hong SB, Lim CM, Koh Y, Kim WY. Lactate Level Versus Lactate Clearance for Predicting Mortality in patients with septic shock defined by Sepsis-3. Crit Care Med. 2018;46(6):e489–95.PubMedCrossRef Ryoo SM, Lee J, Lee YS, Lee JH, Lim KS, Huh JW, Hong SB, Lim CM, Koh Y, Kim WY. Lactate Level Versus Lactate Clearance for Predicting Mortality in patients with septic shock defined by Sepsis-3. Crit Care Med. 2018;46(6):e489–95.PubMedCrossRef
79.
go back to reference Estenssoro E, Kanoore Edul VS, Loudet CI, Osatnik J, Ríos FG, Vázquez DN, Pozo MO, Lattanzio B, Pálizas F, Klein F, et al. Predictive validity of Sepsis-3 definitions and Sepsis outcomes in critically Ill patients: a Cohort Study in 49 ICUs in Argentina. Crit Care Med. 2018;46(8):1276–83.PubMedCrossRef Estenssoro E, Kanoore Edul VS, Loudet CI, Osatnik J, Ríos FG, Vázquez DN, Pozo MO, Lattanzio B, Pálizas F, Klein F, et al. Predictive validity of Sepsis-3 definitions and Sepsis outcomes in critically Ill patients: a Cohort Study in 49 ICUs in Argentina. Crit Care Med. 2018;46(8):1276–83.PubMedCrossRef
80.
go back to reference Mikkelsen ME, Miltiades AN, Gaieski DF, Goyal M, Fuchs BD, Shah CV, Bellamy SL, Christie JD. Serum lactate is associated with mortality in severe sepsis Independent of organ failure and shock. Crit Care Med. 2009;37(5):1670–7.PubMedCrossRef Mikkelsen ME, Miltiades AN, Gaieski DF, Goyal M, Fuchs BD, Shah CV, Bellamy SL, Christie JD. Serum lactate is associated with mortality in severe sepsis Independent of organ failure and shock. Crit Care Med. 2009;37(5):1670–7.PubMedCrossRef
81.
go back to reference Pan J, Peng M, Liao C, Hu X, Wang A, Li X. Relative efficacy and safety of early lactate clearance-guided therapy resuscitation in patients with sepsis: a meta-analysis. Medicine. 2019;98(8):e14453.PubMedPubMedCentralCrossRef Pan J, Peng M, Liao C, Hu X, Wang A, Li X. Relative efficacy and safety of early lactate clearance-guided therapy resuscitation in patients with sepsis: a meta-analysis. Medicine. 2019;98(8):e14453.PubMedPubMedCentralCrossRef
82.
go back to reference Jones AE, Shapiro NI, Trzeciak S, Arnold RC, Claremont HA, Kline JA. Lactate clearance vs central venous oxygen saturation as goals of early sepsis therapy: a randomized clinical trial. JAMA. 2010;303(8):739–46.PubMedPubMedCentralCrossRef Jones AE, Shapiro NI, Trzeciak S, Arnold RC, Claremont HA, Kline JA. Lactate clearance vs central venous oxygen saturation as goals of early sepsis therapy: a randomized clinical trial. JAMA. 2010;303(8):739–46.PubMedPubMedCentralCrossRef
83.
go back to reference Muntner P, Warnock DG. Acute kidney injury in sepsis: questions answered, but others remain. Kidney Int. 2010;77(6):485–7.PubMedCrossRef Muntner P, Warnock DG. Acute kidney injury in sepsis: questions answered, but others remain. Kidney Int. 2010;77(6):485–7.PubMedCrossRef
85.
go back to reference Andrews B, Semler MW, Muchemwa L, Kelly P, Lakhi S, Heimburger DC, Mabula C, Bwalya M, Bernard GR. Effect of an early resuscitation protocol on In-hospital mortality among adults with Sepsis and hypotension: a Randomized Clinical Trial. JAMA. 2017;318(13):1233–40.PubMedPubMedCentralCrossRef Andrews B, Semler MW, Muchemwa L, Kelly P, Lakhi S, Heimburger DC, Mabula C, Bwalya M, Bernard GR. Effect of an early resuscitation protocol on In-hospital mortality among adults with Sepsis and hypotension: a Randomized Clinical Trial. JAMA. 2017;318(13):1233–40.PubMedPubMedCentralCrossRef
Metadata
Title
Machine learning for the prediction of sepsis-related death: a systematic review and meta-analysis
Authors
Yan Zhang
Weiwei Xu
Ping Yang
An Zhang
Publication date
01-12-2023
Publisher
BioMed Central
Published in
BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI
https://doi.org/10.1186/s12911-023-02383-1

Other articles of this Issue 1/2023

BMC Medical Informatics and Decision Making 1/2023 Go to the issue