Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Annals of Intensive Care
Published in: Annals of Intensive Care 1/2017

Open Access 01-12-2017 | Research

External validation of SAPS 3 and MPM0-III scores in 48,816 patients from 72 Brazilian ICUs

Authors: Giulliana Martines Moralez, Ligia Sarmet Cunha Farah Rabello, Thiago Costa Lisboa, Mariza da Fonte Andrade Lima, Rodrigo Marques Hatum, Fernando Vinicius Cesar De Marco, Alessandra Alves, Jorge Eduardo da Silva Soares Pinto, Hélia Beatriz Nunes de Araújo, Grazielle Viana Ramos, Aline Reis Silva, Guilherme Côrtes Fernandes, Guilherme Brenande Alves Faria, Ciro Leite Mendes, Roberto Álvaro Ramos Filho, Valdênia Pereira de Souza, Pedro Emmanuel Alvarenga Americano do Brasil, Fernando Augusto Bozza, Jorge Ibrain Figueira Salluh, Marcio Soares, On Behalf of the ORCHESTRA Study Investigators

Published in: Annals of Intensive Care | Issue 1/2017

Login to get access

Abstract

Background

The performance of severity-of-illness scores varies in different scenarios and must be validated prior of being used in a specific settings and geographic regions. Moreover, models’ calibration may deteriorate overtime and performance of such instruments should be reassessed regularly. Therefore, we aimed at to validate the SAPS 3 in a large contemporary cohort of patients admitted to Brazilian ICUs. In addition, we also compared the performance of the SAPS 3 with the MPM0-III.

Methods

This is a retrospective cohort study in which 48,816 (medical admissions = 67.9%) adult patients are admitted to 72 Brazilian ICUs during 2013. We evaluated models’ discrimination using the area under the receiver operating characteristic curve (AUROC). We applied the calibration belt to evaluate the agreement between observed and expected mortality rates (calibration).

Results

Mean SAPS 3 score was 44.3 ± 15.4 points. ICU and hospital mortality rates were 11.0 and 16.5%. We estimated predicted mortality using both standard (SE) and Central and South American (CSA) customized equations. Predicted mortality rates were 16.4 ± 19.3% (SAPS 3-SE), 21.7 ± 23.2% (SAPS 3-CSA) and 14.3 ± 14.0% (MPM0-III). Standardized mortality ratios (SMR) obtained for each model were: 1.00 (95% CI, 0.98–0.102) for the SAPS 3-SE, 0.75 (0.74–0.77) for the SAPS 3-CSA and 1.15 (1.13–1.18) for the MPM0-III. Discrimination was better for SAPS 3 models (AUROC = 0.85) than for MPM0-III (AUROC = 0.80) (p < 0.001). We applied the calibration belt to evaluate the agreement between observed and expected mortality rates (calibration): the SAPS 3-CSA overestimated mortality throughout all risk classes while the MPM0-III underestimated it uniformly. The SAPS 3-SE did not show relevant deviations from ideal calibration.

Conclusions

In a large contemporary database, the SAPS 3-SE was accurate in predicting outcomes, supporting its use for performance evaluation and benchmarking in Brazilian ICUs.
Appendix
Available only for authorised users
Literature
1.
Salluh JIF, Soares M. ICU severity of illness scores: APACHE, SAPS and MPM. Curr Opin Crit Care. 2014;20(5):557–65.CrossRefPubMed
2.
Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34(5):1297–310.CrossRefPubMed
3.
Higgins TL, Teres D, Copes WS, Nathanson BH, Stark M, Kramer AA. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). Crit Care Med. 2007;35(3):827–35.CrossRefPubMed
4.
Metnitz PGH, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3: from evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohort description. Intensive Care Med. 2005;31(10):1336–44.CrossRefPubMedPubMedCentral
5.
Moreno RP, Metnitz PGH, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3: from evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345–55.CrossRefPubMedPubMedCentral
6.
7.
de Oliveira VM, Brauner JS, Rodrigues Filho E, Susin RGA, Draghetti V, Bolzan ST, et al. Is SAPS 3 better than APACHE II at predicting mortality in critically ill transplant patients? Clin Sao Paulo Braz. 2013;68(2):153–8.CrossRef
8.
Soares M, Salluh JIF. Validation of the SAPS 3 admission prognostic model in patients with cancer in need of intensive care. Intensive Care Med. 2006;32(11):1839–44.CrossRefPubMed
9.
Soares M, Silva UVA, Teles JMM, Silva E, Caruso P, Lobo SMA, et al. Validation of four prognostic scores in patients with cancer admitted to Brazilian intensive care units: results from a prospective multicenter study. Intensive Care Med. 2010;36(7):1188–95.CrossRefPubMed
10.
Costa e Silva VT, de Castro I, Liano F, Muriel A, Rodriguez-Palomares JR, Yu L. Performance of the third-generation models of severity scoring systems (APACHE IV, SAPS 3 and MPM-III) in acute kidney injury critically ill patients. Nephrol Dial Transplant. 2011;26(12):3894–901.CrossRefPubMed
11.
Nassar Junior AP, Mocelin AO, Andrade FM, Brauer L, Giannini FP, Nunes ALB, et al. SAPS 3, APACHE IV or GRACE: which score to choose for acute coronary syndrome patients in intensive care units? Sao Paulo Med J Rev Paul Med. 2013;131(3):173–8.CrossRef
12.
Maccariello ER, Valente C, Nogueira L, Ismael M, Valenca RVR, Machado JES, et al. Performance of six prognostic scores in critically ILL patients receiving renal replacement therapy. Rev Bras Ter Intensiva. 2008;20(2):115–23.CrossRefPubMed
13.
Alves CJ, Franco GPP, Nakata CT, Costa GLG, Costa GLG, Genaro MS, et al. Evaluation of prognostic indicators for elderly patients admitted in intensive care units. Rev Bras Ter Intensiva. 2009;21(1):1–8.CrossRefPubMed
14.
Serpa Neto A, de Assuncao MSC, Pardini A, Silva E. Feasibility of transitioning from APACHE II to SAPS III as prognostic model in a Brazilian general intensive care unit. A retrospective study. Sao Paulo Med. J Rev Paul Med. 2015;133(3):199–205.CrossRef
15.
Silva Junior JM, Malbouisson LMS, Nuevo HL, Barbosa LGT, Marubayashi LY, Teixeira IC, et al. Applicability of the simplified acute physiology score (SAPS 3) in Brazilian hospitals. Rev Bras Anestesiol. 2010;60(1):20–31.CrossRefPubMed
16.
Nassar APJ, Mocelin AO, Nunes ALB, Giannini FP, Brauer L, Andrade FM, et al. Caution when using prognostic models: a prospective comparison of 3 recent prognostic models. J Crit Care. 2012;27(4):423.e1–7.CrossRef
17.
Keegan MT, Gajic O, Afessa B. Severity of illness scoring systems in the intensive care unit. Crit Care Med. 2011;39(1):163–9.CrossRefPubMed
18.
Soares M, Bozza FA, Angus DC, Japiassu AM, Viana WN, Costa R, et al. Organizational characteristics, outcomes, and resource use in 78 Brazilian intensive care units: the ORCHESTRA study. Intensive Care Med. 2015;41(12):2149–60.CrossRefPubMed
19.
Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83.CrossRefPubMed
20.
Oken MM, Creech RH, Tormey DC, Horton J, Davis TE, McFadden ET, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982;5(6):649–55.CrossRefPubMed
21.
Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–10.CrossRefPubMed
22.
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.CrossRefPubMed
23.
Finazzi S, Poole D, Luciani D, Cogo PE, Bertolini G. Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLoS ONE. 2011;6(2):e16110.CrossRefPubMedPubMedCentral
24.
Poole D, Rossi C, Latronico N, Rossi G, Finazzi S, Bertolini G. Comparison between SAPS II and SAPS 3 in predicting hospital mortality in a cohort of 103 Italian ICUs. Is new always better? Intensive Care Med. 2012;38(8):1280–8.CrossRefPubMed
25.
Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: The Hosmer–Lemeshow test revisited. Crit Care Med. 2007;35(9):2052–6.CrossRefPubMed
26.
Peek N, Arts DGT, Bosman RJ, van der Voort PHJ, de Keizer NF. External validation of prognostic models for critically ill patients required substantial sample sizes. J Clin Epidemiol. 2007;60(5):491–501.CrossRefPubMed
27.
Reineck LA, Pike F, Le TQ, Cicero BD, Iwashyna TJ, Kahn JM. Hospital factors associated with discharge bias in ICU performance measurement. Crit Care Med. 2014;42(5):1055–64.CrossRefPubMed
Metadata
Title
External validation of SAPS 3 and MPM0-III scores in 48,816 patients from 72 Brazilian ICUs
Authors
Giulliana Martines Moralez
Ligia Sarmet Cunha Farah Rabello
Thiago Costa Lisboa
Mariza da Fonte Andrade Lima
Rodrigo Marques Hatum
Fernando Vinicius Cesar De Marco
Alessandra Alves
Jorge Eduardo da Silva Soares Pinto
Hélia Beatriz Nunes de Araújo
Grazielle Viana Ramos
Aline Reis Silva
Guilherme Côrtes Fernandes
Guilherme Brenande Alves Faria
Ciro Leite Mendes
Roberto Álvaro Ramos Filho
Valdênia Pereira de Souza
Pedro Emmanuel Alvarenga Americano do Brasil
Fernando Augusto Bozza
Jorge Ibrain Figueira Salluh
Marcio Soares
On Behalf of the ORCHESTRA Study Investigators
Publication date
01-12-2017
Publisher
Springer Paris
Published in
Annals of Intensive Care / Issue 1/2017
Electronic ISSN: 2110-5820
DOI
https://doi.org/10.1186/s13613-017-0276-3

Other articles of this Issue 1/2017

Annals of Intensive Care 1/2017 Go to the issue

Research

Acute hyperventilation increases the central venous-to-arterial PCO2 difference in stable septic shock patients

Review

Nebulized antibiotics in mechanically ventilated patients: a challenge for translational research from technology to clinical care

Research

Systematic overdosing of oxa- and cloxacillin in severe infections treated in ICU: risk factors and side effects

Research

Invasive fungal tracheobronchitis in mechanically ventilated critically ill patients: underlying conditions, diagnosis, and outcomes

Research

Associations among left ventricular systolic function, tachycardia, and cardiac preload in septic patients

Research

Systemic antibiotics for preventing ventilator-associated pneumonia in comatose patients: a systematic review and meta-analysis