Skip to main content
Top
Published in: Journal of Translational Medicine 1/2021

01-12-2021 | Septicemia | Research

Development and validation of a score to predict mortality in ICU patients with sepsis: a multicenter retrospective study

Authors: Jie Weng, Ruonan Hou, Xiaoming Zhou, Zhe Xu, Zhiliang Zhou, Peng Wang, Liang Wang, Chan Chen, Jinyu Wu, Zhiyi Wang

Published in: Journal of Translational Medicine | Issue 1/2021

Login to get access

Abstract

Background

Early and accurate identification of septic patients at high risk for ICU mortality can help clinicians make optimal clinical decisions and improve the patients’ outcomes. This study aimed to develop and validate (internally and externally) a mortality prediction score for sepsis following admission in the ICU.

Methods

We extracted data retrospectively regarding adult septic patients from one teaching hospital in Wenzhou, China and a large multi-center critical care database from the USA. Demographic data, vital signs, laboratory values, comorbidities, and clinical outcomes were collected. The primary outcome was ICU mortality. Through multivariable logistic regression, a mortality prediction score for sepsis was developed and validated.

Results

Four thousand two hundred and thirty six patients in the development cohort and 8359 patients in three validation cohorts. The Prediction of Sepsis Mortality in ICU (POSMI) score included age ≥ 50 years, temperature < 37 °C, Respiratory rate > 35 breaths/min, MAP ≤ 50 mmHg, SpO2 < 90%, albumin ≤ 2 g/dL, bilirubin ≥ 0.8 mg/dL, lactate ≥ 4.2 mmol/L, BUN ≥ 21 mg/dL, mechanical ventilation, hepatic failure and metastatic cancer. In addition, the area under the receiver operating characteristic curve (AUC) for the development cohort was 0.831 (95% CI, 0.813–0.850) while the AUCs ranged from 0.798 to 0.829 in the three validation cohorts. Moreover, the POSMI score had a higher AUC than both the SOFA and APACHE IV scores. Notably, the Hosmer–Lemeshow (H–L) goodness-of-fit test results and calibration curves suggested good calibration in the development and validation cohorts. Additionally, the POSMI score still exhibited excellent discrimination and calibration following sensitivity analysis. With regard to clinical usefulness, the decision curve analysis (DCA) of POSMI showed a higher net benefit than SOFA and APACHE IV in the development cohort.

Conclusion

POSMI was validated to be an effective tool for predicting mortality in ICU patients with sepsis.
Appendix
Available only for authorised users
Literature
1.
go back to reference Vincent JL, Marshall JC, Namendys-Silva SA, et al. Assessment of the worldwide burden of critical illness: the intensive care over nations (ICON) audit. Lancet Respir Med. 2014;2(5):380–6.PubMedCrossRef Vincent JL, Marshall JC, Namendys-Silva SA, et al. Assessment of the worldwide burden of critical illness: the intensive care over nations (ICON) audit. Lancet Respir Med. 2014;2(5):380–6.PubMedCrossRef
2.
go back to reference Lelubre C, Vincent JL. Mechanisms and treatment of organ failure in sepsis. Nat Rev Nephrol. 2018;14(7):417–27.PubMedCrossRef Lelubre C, Vincent JL. Mechanisms and treatment of organ failure in sepsis. Nat Rev Nephrol. 2018;14(7):417–27.PubMedCrossRef
3.
go back to reference Yébenes JC, Ruiz-Rodriguez JC, Ferrer R, et al. Epidemiology of sepsis in Catalonia: analysis of incidence and outcomes in a European setting. Ann Intensive Care. 2017;7(1):19.PubMedPubMedCentralCrossRef Yébenes JC, Ruiz-Rodriguez JC, Ferrer R, et al. Epidemiology of sepsis in Catalonia: analysis of incidence and outcomes in a European setting. Ann Intensive Care. 2017;7(1):19.PubMedPubMedCentralCrossRef
4.
go back to reference Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–10.PubMedPubMedCentralCrossRef Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–10.PubMedPubMedCentralCrossRef
5.
go back to reference Knaus WA, Draper EA, Wagner DP, et al. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.CrossRefPubMed Knaus WA, Draper EA, Wagner DP, et al. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.CrossRefPubMed
6.
go back to reference Zimmerman JE, Kramer AA, McNair DS, et al. Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34(5):1297–310.PubMedCrossRef Zimmerman JE, Kramer AA, McNair DS, et al. Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34(5):1297–310.PubMedCrossRef
7.
go back to reference Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–63.PubMedCrossRef Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–63.PubMedCrossRef
8.
go back to reference Moreno RP, Metnitz PG, Almeida E, et al. SAPS 3—from evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345–55.PubMedPubMedCentralCrossRef Moreno RP, Metnitz PG, Almeida E, et al. SAPS 3—from evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345–55.PubMedPubMedCentralCrossRef
9.
go back to reference Cardoso T, Teixeira-Pinto A, Rodrigues PP, et al. Predisposition, insult/infection, response and organ dysfunction (PIRO): a pilot clinical staging system for hospital mortality in patients with infection. PLoS ONE. 2013;8(7):e70806.PubMedPubMedCentralCrossRef Cardoso T, Teixeira-Pinto A, Rodrigues PP, et al. Predisposition, insult/infection, response and organ dysfunction (PIRO): a pilot clinical staging system for hospital mortality in patients with infection. PLoS ONE. 2013;8(7):e70806.PubMedPubMedCentralCrossRef
10.
go back to reference Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83.PubMedCrossRef Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83.PubMedCrossRef
11.
go back to reference Ghorbani M, Ghaem H, Rezaianzadeh A, et al. A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran. F1000Res. 2017;6:2032.PubMedPubMedCentralCrossRef Ghorbani M, Ghaem H, Rezaianzadeh A, et al. A study on the efficacy of APACHE-IV for predicting mortality and length of stay in an intensive care unit in Iran. F1000Res. 2017;6:2032.PubMedPubMedCentralCrossRef
12.
go back to reference Pollard TJ, Johnson AEW, Raffa JD, et al. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Sci Data. 2018;5:180178.PubMedPubMedCentralCrossRef Pollard TJ, Johnson AEW, Raffa JD, et al. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Sci Data. 2018;5:180178.PubMedPubMedCentralCrossRef
13.
go back to reference O’Halloran HM, Kwong K, Veldhoen RA, Maslove DM. Characterizing the patients, hospitals, and data quality of the eICU collaborative research database. Crit Care Med. 2020;48(12):1737–43.PubMedCrossRef O’Halloran HM, Kwong K, Veldhoen RA, Maslove DM. Characterizing the patients, hospitals, and data quality of the eICU collaborative research database. Crit Care Med. 2020;48(12):1737–43.PubMedCrossRef
14.
go back to reference Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):775–87.PubMedPubMedCentralCrossRef Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):775–87.PubMedPubMedCentralCrossRef
15.
go back to reference Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS international sepsis definitions conference. Crit Care Med. 2003;31(4):1250–6.PubMedCrossRef Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS international sepsis definitions conference. Crit Care Med. 2003;31(4):1250–6.PubMedCrossRef
16.
go back to reference Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87.PubMedCrossRef Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87.PubMedCrossRef
17.
go back to reference DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.PubMedCrossRef DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.PubMedCrossRef
18.
go back to reference Pencina MJ, D’Agostino RB Sr, D’Agostino RB, et al. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72.PubMedCrossRef Pencina MJ, D’Agostino RB Sr, D’Agostino RB, et al. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72.PubMedCrossRef
19.
go back to reference Serafim R, Gomes JA, Salluh J, et al. A comparison of the Quick-SOFA and systemic inflammatory response syndrome criteria for the diagnosis of sepsis and prediction of mortality: a systematic review and meta-analysis. Chest. 2018;153(3):646–55.PubMedCrossRef Serafim R, Gomes JA, Salluh J, et al. A comparison of the Quick-SOFA and systemic inflammatory response syndrome criteria for the diagnosis of sepsis and prediction of mortality: a systematic review and meta-analysis. Chest. 2018;153(3):646–55.PubMedCrossRef
20.
go back to reference Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):762–74.PubMedPubMedCentralCrossRef Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):762–74.PubMedPubMedCentralCrossRef
21.
go back to reference Ford DW, Goodwin AJ, Simpson AN, et al. A severe sepsis mortality prediction model and score for use with administrative data. Crit Care Med. 2016;44(2):319–27.PubMedPubMedCentralCrossRef Ford DW, Goodwin AJ, Simpson AN, et al. A severe sepsis mortality prediction model and score for use with administrative data. Crit Care Med. 2016;44(2):319–27.PubMedPubMedCentralCrossRef
22.
go back to reference Liu Z, Meng Z, Li Y, et al. Prognostic accuracy of the serum lactate level, the SOFA score and the qSOFA score for mortality among adults with Sepsis. Scand J Trauma Resusc Emerg Med. 2019;27(1):51.PubMedPubMedCentralCrossRef Liu Z, Meng Z, Li Y, et al. Prognostic accuracy of the serum lactate level, the SOFA score and the qSOFA score for mortality among adults with Sepsis. Scand J Trauma Resusc Emerg Med. 2019;27(1):51.PubMedPubMedCentralCrossRef
23.
go back to reference Wernly B, Mamandipoor B, Baldia P, et al. Machine learning predicts mortality in septic patients using only routinely available ABG variables: a multi-centre evaluation. Int J Med Inform. 2021;145:104312.PubMedCrossRef Wernly B, Mamandipoor B, Baldia P, et al. Machine learning predicts mortality in septic patients using only routinely available ABG variables: a multi-centre evaluation. Int J Med Inform. 2021;145:104312.PubMedCrossRef
24.
go back to reference Raith EP, Udy AA, Bailey M, et al. Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317(3):290–300.PubMedCrossRef Raith EP, Udy AA, Bailey M, et al. Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317(3):290–300.PubMedCrossRef
25.
go back to reference Zhang XM, Zhang WW, Yu XZ, et al. Comparing the performance of SOFA, TPA combined with SOFA and APACHE-II for predicting ICU mortality in critically ill surgical patients: a secondary analysis. Clin Nutr. 2020;39(9):2902–9.PubMedCrossRef Zhang XM, Zhang WW, Yu XZ, et al. Comparing the performance of SOFA, TPA combined with SOFA and APACHE-II for predicting ICU mortality in critically ill surgical patients: a secondary analysis. Clin Nutr. 2020;39(9):2902–9.PubMedCrossRef
26.
go back to reference Gao X, Liu Y, Xu F, et al. Assessment of apoptosis inhibitor of macrophage/CD5L as a biomarker to predict mortality in the critically ill with sepsis. Chest. 2019;156(4):696–705.PubMedCrossRef Gao X, Liu Y, Xu F, et al. Assessment of apoptosis inhibitor of macrophage/CD5L as a biomarker to predict mortality in the critically ill with sepsis. Chest. 2019;156(4):696–705.PubMedCrossRef
27.
go back to reference Bennett CE, Wright RS, Jentzer J, et al. Severity of illness assessment with application of the APACHE IV predicted mortality and outcome trends analysis in an academic cardiac intensive care unit. J Crit Care. 2019;50:242–6.PubMedCrossRef Bennett CE, Wright RS, Jentzer J, et al. Severity of illness assessment with application of the APACHE IV predicted mortality and outcome trends analysis in an academic cardiac intensive care unit. J Crit Care. 2019;50:242–6.PubMedCrossRef
28.
29.
go back to reference Wu DY, Lu SQ. The effects of abnormal body temperature on the prognosis of patients with septic shock. Ther Hypothermia Temp Manag. 2020;10(3):148–52.PubMedCrossRef Wu DY, Lu SQ. The effects of abnormal body temperature on the prognosis of patients with septic shock. Ther Hypothermia Temp Manag. 2020;10(3):148–52.PubMedCrossRef
31.
go back to reference Schortgen F, Clabault K, Katsahian S, et al. Fever control using external cooling in septic shock: a randomized controlled trial. Am J Respir Crit Care Med. 2012;185(10):1088–95.PubMedCrossRef Schortgen F, Clabault K, Katsahian S, et al. Fever control using external cooling in septic shock: a randomized controlled trial. Am J Respir Crit Care Med. 2012;185(10):1088–95.PubMedCrossRef
32.
go back to reference Rumbus Z, Matics R, Hegyi P, et al. Fever is associated with reduced, hypothermia with increased mortality in septic patients: a meta-analysis of clinical trials. PLoS ONE. 2017;12(1):e0170152.PubMedPubMedCentralCrossRef Rumbus Z, Matics R, Hegyi P, et al. Fever is associated with reduced, hypothermia with increased mortality in septic patients: a meta-analysis of clinical trials. PLoS ONE. 2017;12(1):e0170152.PubMedPubMedCentralCrossRef
34.
go back to reference Kushimoto S, Abe T, Ogura H, et al. Impact of body temperature abnormalities on the implementation of sepsis bundles and outcomes in patients with severe sepsis: a retrospective sub-analysis of the focused outcome research on emergency care for acute respiratory distress syndrome, sepsis and trauma study. Crit Care Med. 2019;47(5):691–9.PubMedCrossRef Kushimoto S, Abe T, Ogura H, et al. Impact of body temperature abnormalities on the implementation of sepsis bundles and outcomes in patients with severe sepsis: a retrospective sub-analysis of the focused outcome research on emergency care for acute respiratory distress syndrome, sepsis and trauma study. Crit Care Med. 2019;47(5):691–9.PubMedCrossRef
36.
go back to reference Arbo JE, Lessing JK, Ford WJH, et al. Heart rate variability measures for prediction of severity of illness and poor outcome in ED patients with sepsis. Am J Emerg Med. 2020;38(12):2607–13.PubMedCrossRefPubMedCentral Arbo JE, Lessing JK, Ford WJH, et al. Heart rate variability measures for prediction of severity of illness and poor outcome in ED patients with sepsis. Am J Emerg Med. 2020;38(12):2607–13.PubMedCrossRefPubMedCentral
37.
go back to reference Jarkovska D, Valesova L, Chvojka J, et al. Heart-rate variability depression in porcine peritonitis-induced sepsis without organ failure. Exp Biol Med (Maywood). 2017;242(9):1005–12.CrossRef Jarkovska D, Valesova L, Chvojka J, et al. Heart-rate variability depression in porcine peritonitis-induced sepsis without organ failure. Exp Biol Med (Maywood). 2017;242(9):1005–12.CrossRef
38.
go back to reference Barnaby DP, Fernando SM, Herry CL, et al. Heart rate variability, clinical and laboratory measures to predict future deterioration in patients presenting with sepsis. Shock. 2019;51(4):416–22.PubMedCrossRef Barnaby DP, Fernando SM, Herry CL, et al. Heart rate variability, clinical and laboratory measures to predict future deterioration in patients presenting with sepsis. Shock. 2019;51(4):416–22.PubMedCrossRef
39.
go back to reference von Bahr V, Hultman J, Eksborg S, et al. Long-term survival in adults treated with extracorporeal membrane oxygenation for respiratory failure and Sepsis. Crit Care Med. 2017;45(2):164–70.CrossRef von Bahr V, Hultman J, Eksborg S, et al. Long-term survival in adults treated with extracorporeal membrane oxygenation for respiratory failure and Sepsis. Crit Care Med. 2017;45(2):164–70.CrossRef
40.
41.
go back to reference Jensen JS, Peters L, Itenov TS, et al. Biomarker-assisted identification of sepsis-related acute liver impairment: a frequent and deadly condition in critically ill patients. Clin Chem Lab Med. 2019;57(9):1422–31.PubMedCrossRef Jensen JS, Peters L, Itenov TS, et al. Biomarker-assisted identification of sepsis-related acute liver impairment: a frequent and deadly condition in critically ill patients. Clin Chem Lab Med. 2019;57(9):1422–31.PubMedCrossRef
42.
go back to reference Ryoo SM, Lee J, Lee YS, et al. Lactate level versus lactate clearance for predicting mortality in patients with septic shock defined by sepsis-3. Crit Care Med. 2018;46(6):e489–95.PubMedCrossRef Ryoo SM, Lee J, Lee YS, et al. Lactate level versus lactate clearance for predicting mortality in patients with septic shock defined by sepsis-3. Crit Care Med. 2018;46(6):e489–95.PubMedCrossRef
43.
go back to reference Mikkelsen ME, Miltiades AN, Gaieski DF, et al. Serum lactate is associated with mortality in severe sepsis independent of organ failure and shock. Crit Care Med. 2009;37(5):1670–7.PubMedCrossRef Mikkelsen ME, Miltiades AN, Gaieski DF, et al. Serum lactate is associated with mortality in severe sepsis independent of organ failure and shock. Crit Care Med. 2009;37(5):1670–7.PubMedCrossRef
44.
go back to reference Woolum JA, Abner EL, Kelly A, Thompson Bastin ML, Morris PE, Flannery AH. Effect of thiamine administration on lactate clearance and mortality in patients with septic shock. Crit Care Med. 2018;46(11):1747–52.PubMedCrossRef Woolum JA, Abner EL, Kelly A, Thompson Bastin ML, Morris PE, Flannery AH. Effect of thiamine administration on lactate clearance and mortality in patients with septic shock. Crit Care Med. 2018;46(11):1747–52.PubMedCrossRef
Metadata
Title
Development and validation of a score to predict mortality in ICU patients with sepsis: a multicenter retrospective study
Authors
Jie Weng
Ruonan Hou
Xiaoming Zhou
Zhe Xu
Zhiliang Zhou
Peng Wang
Liang Wang
Chan Chen
Jinyu Wu
Zhiyi Wang
Publication date
01-12-2021
Publisher
BioMed Central
Published in
Journal of Translational Medicine / Issue 1/2021
Electronic ISSN: 1479-5876
DOI
https://doi.org/10.1186/s12967-021-03005-y

Other articles of this Issue 1/2021

Journal of Translational Medicine 1/2021 Go to the issue
Live Webinar | 27-06-2024 | 18:00 (CEST)

Keynote webinar | Spotlight on medication adherence

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

WHO estimates that half of all patients worldwide are non-adherent to their prescribed medication. The consequences of poor adherence can be catastrophic, on both the individual and population level.

Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.

Prof. Kevin Dolgin
Prof. Florian Limbourg
Prof. Anoop Chauhan
Developed by: Springer Medicine
Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine

Highlights from the ACC 2024 Congress

Year in Review: Pediatric cardiology

Watch Dr. Anne Marie Valente present the last year's highlights in pediatric and congenital heart disease in the official ACC.24 Year in Review session.

Year in Review: Pulmonary vascular disease

The last year's highlights in pulmonary vascular disease are presented by Dr. Jane Leopold in this official video from ACC.24.

Year in Review: Valvular heart disease

Watch Prof. William Zoghbi present the last year's highlights in valvular heart disease from the official ACC.24 Year in Review session.

Year in Review: Heart failure and cardiomyopathies

Watch this official video from ACC.24. Dr. Biykem Bozkurt discusses last year's major advances in heart failure and cardiomyopathies.