Skip to main content
Top
Published in: Critical Care 6/2005

Open Access 01-12-2005 | Research

Mortality prediction using SAPS II: an update for French intensive care units

Authors: Jean Roger Le Gall, Anke Neumann, François Hemery, Jean Pierre Bleriot, Jean Pierre Fulgencio, Bernard Garrigues, Christian Gouzes, Eric Lepage, Pierre Moine, Daniel Villers

Published in: Critical Care | Issue 6/2005

Login to get access

Abstract

Introduction

The standardized mortality ratio (SMR) is commonly used for benchmarking intensive care units (ICUs). Available mortality prediction models are outdated and must be adapted to current populations of interest. The objective of this study was to improve the Simplified Acute Physiology Score (SAPS) II for mortality prediction in ICUs, thereby improving SMR estimates.

Method

A retrospective data base study was conducted in patients hospitalized in 106 French ICUs between 1 January 1998 and 31 December 1999. A total of 77,490 evaluable admissions were split into a training set and a validation set. Calibration and discrimination were determined for the original SAPS II, a customized SAPS II and an expanded SAPS II developed in the training set by adding six admission variables: age, sex, length of pre-ICU hospital stay, patient location before ICU, clinical category and whether drug overdose was present. The training set was used for internal validation and the validation set for external validation.

Results

With the original SAPS II calibration was poor, with marked underestimation of observed mortality, whereas discrimination was good (area under the receiver operating characteristic curve 0.858). Customization improved calibration but had poor uniformity of fit; discrimination was unchanged. The expanded SAPS II exhibited good calibration, good uniformity of fit and better discrimination (area under the receiver operating characteristic curve 0.879). The SMR in the validation set was 1.007 (confidence interval 0.985–1.028). Some ICUs had better and others worse performance with the expanded SAPS II than with the customized SAPS II.

Conclusion

The original SAPS II model did not perform sufficiently well to be useful for benchmarking in France. Customization improved the statistical qualities of the model but gave poor uniformity of fit. Adding simple variables to create an expanded SAPS II model led to better calibration, discrimination and uniformity of fit, producing a tool suitable for benchmarking.
Appendix
Available only for authorised users
Literature
1.
go back to reference Le Gall JR, Loirat P: Can we evaluate the performance of an intensive care unit? Curr Opin Crit Care 1995, 1: 219-220.CrossRef Le Gall JR, Loirat P: Can we evaluate the performance of an intensive care unit? Curr Opin Crit Care 1995, 1: 219-220.CrossRef
2.
go back to reference Ridley S: Severity of illness scoring systems and performance appraisal. Anaesthesia 1998, 53: 1185-1194. 10.1046/j.1365-2044.1998.00615.xCrossRefPubMed Ridley S: Severity of illness scoring systems and performance appraisal. Anaesthesia 1998, 53: 1185-1194. 10.1046/j.1365-2044.1998.00615.xCrossRefPubMed
3.
go back to reference Lemeshow S, Le Gall JR: Modeling the severity of illness of ICU patients. A system update. JAMA 1994, 272: 1049-1055. 10.1001/jama.272.13.1049CrossRefPubMed Lemeshow S, Le Gall JR: Modeling the severity of illness of ICU patients. A system update. JAMA 1994, 272: 1049-1055. 10.1001/jama.272.13.1049CrossRefPubMed
4.
go back to reference Le Gall JR, Lemeshow S, Saulnier F: A new Simplified Acute Physiologic Score (SAPS II) based on an European/North American multicenter study. JAMA 1993, 270: 2957-2963. Correction: JAMA 1994, 271: 1321. 10.1001/jama.270.24.2957CrossRefPubMed Le Gall JR, Lemeshow S, Saulnier F: A new Simplified Acute Physiologic Score (SAPS II) based on an European/North American multicenter study. JAMA 1993, 270: 2957-2963. Correction: JAMA 1994, 271: 1321. 10.1001/jama.270.24.2957CrossRefPubMed
5.
go back to reference Glance LG, Osler TM, Dick A: Rating the quality of intensive care units: is it a function of the intensive care unit scoring system? Crit Care Med 2002, 30: 1976-1982. 10.1097/00003246-200209000-00005CrossRefPubMed Glance LG, Osler TM, Dick A: Rating the quality of intensive care units: is it a function of the intensive care unit scoring system? Crit Care Med 2002, 30: 1976-1982. 10.1097/00003246-200209000-00005CrossRefPubMed
6.
go back to reference Moreno R, Apolone G: Impact of different customization strategies in the performance of a general severity score. Crit Care Med 1997, 25: 2001-2008. 10.1097/00003246-199712000-00017CrossRefPubMed Moreno R, Apolone G: Impact of different customization strategies in the performance of a general severity score. Crit Care Med 1997, 25: 2001-2008. 10.1097/00003246-199712000-00017CrossRefPubMed
7.
go back to reference Metnitz PG, Lang T, Vesely H, Valentin A, Le Gall JR: Ratios of observed to expected mortality are affected by difference in case mix and quality of care. Intensive Care Med 2000, 26: 1466-1472. 10.1007/s001340000638CrossRefPubMed Metnitz PG, Lang T, Vesely H, Valentin A, Le Gall JR: Ratios of observed to expected mortality are affected by difference in case mix and quality of care. Intensive Care Med 2000, 26: 1466-1472. 10.1007/s001340000638CrossRefPubMed
8.
go back to reference Watson WA, Litovitz TL, Klein-Schwartz W, Rodgers GC Jr, Youniss J, Reid N, Rouse WG, Rembert RS, Borys D: 2003 annual report of the American Association of Poison Control Centers Toxic Exposure Surveillance System. Am J Emerg Med 2004, 22: 335-404. 10.1016/j.ajem.2004.06.001CrossRefPubMed Watson WA, Litovitz TL, Klein-Schwartz W, Rodgers GC Jr, Youniss J, Reid N, Rouse WG, Rembert RS, Borys D: 2003 annual report of the American Association of Poison Control Centers Toxic Exposure Surveillance System. Am J Emerg Med 2004, 22: 335-404. 10.1016/j.ajem.2004.06.001CrossRefPubMed
9.
go back to reference Le Gall JR, Lemeshow S, Leleu G, Klar J, Huillard J, Rue M, Teres D, Artigas A: Customized probability models for early severe sepsis in adult intensive care patients. Intensive Care Unit Scoring Group. JAMA 1995, 273: 644-650. 10.1001/jama.273.8.644CrossRefPubMed Le Gall JR, Lemeshow S, Leleu G, Klar J, Huillard J, Rue M, Teres D, Artigas A: Customized probability models for early severe sepsis in adult intensive care patients. Intensive Care Unit Scoring Group. JAMA 1995, 273: 644-650. 10.1001/jama.273.8.644CrossRefPubMed
10.
go back to reference Hosmer DW, Lemeshow S: Applied Logistic Regression. 2nd edition. New York, NY: John Wiley & Sons; 1989. Hosmer DW, Lemeshow S: Applied Logistic Regression. 2nd edition. New York, NY: John Wiley & Sons; 1989.
11.
go back to reference Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143: 29-36.CrossRefPubMed Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143: 29-36.CrossRefPubMed
12.
go back to reference Hanley JA, Haijan-Tilaki KO: Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: an update. Acad Radiol 1997, 4: 49-58. 10.1016/S1076-6332(97)80161-4CrossRefPubMed Hanley JA, Haijan-Tilaki KO: Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: an update. Acad Radiol 1997, 4: 49-58. 10.1016/S1076-6332(97)80161-4CrossRefPubMed
14.
go back to reference Hastie T, Tibshirani R, Friedman J: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York, NY: Springer-Verlag; 2001.CrossRef Hastie T, Tibshirani R, Friedman J: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York, NY: Springer-Verlag; 2001.CrossRef
15.
go back to reference Breslow NE, Day NE: Statistical Methods in Cancer Research. The Design and Analysis of Cohort Studies. Volume II. Lyon, France: International Agency for Research on Cancer; 1987. Breslow NE, Day NE: Statistical Methods in Cancer Research. The Design and Analysis of Cohort Studies. Volume II. Lyon, France: International Agency for Research on Cancer; 1987.
16.
go back to reference Knaus WA, Draper EA, Wagner DP, Zimmerman JE: APACHE II: a severity of disease classification system. Crit Care Med 1985, 13: 818-829.CrossRefPubMed Knaus WA, Draper EA, Wagner DP, Zimmerman JE: APACHE II: a severity of disease classification system. Crit Care Med 1985, 13: 818-829.CrossRefPubMed
17.
go back to reference Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A, et al.: The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991, 100: 1619-1636.CrossRefPubMed Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A, et al.: The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991, 100: 1619-1636.CrossRefPubMed
18.
go back to reference Lemeshow S, Terres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J: Mortality Probability Models (MPH II) based on an international cohort of intensive care unit patients. JAMA 1993, 270: 2478-2486. 10.1001/jama.270.20.2478CrossRefPubMed Lemeshow S, Terres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J: Mortality Probability Models (MPH II) based on an international cohort of intensive care unit patients. JAMA 1993, 270: 2478-2486. 10.1001/jama.270.20.2478CrossRefPubMed
19.
go back to reference Knaus WA, Harrell FE, Fisher CJ Jr, Wagner DP, Opal SM, Sadoff JC, Draper EA, Walawander CA, Conboy K, Grasela TH: The clinical evaluation of new drugs for sepsis. A prospective study design based on survival analysis. JAMA 1993, 270: 1233-1240. 10.1001/jama.270.10.1233CrossRefPubMed Knaus WA, Harrell FE, Fisher CJ Jr, Wagner DP, Opal SM, Sadoff JC, Draper EA, Walawander CA, Conboy K, Grasela TH: The clinical evaluation of new drugs for sepsis. A prospective study design based on survival analysis. JAMA 1993, 270: 1233-1240. 10.1001/jama.270.10.1233CrossRefPubMed
20.
go back to reference Metnitz PhGH, Moreno RP, Almeida E, Jordan B, Bauer P, Abizanda-Campos R, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, on behalf of the SAPS 3 investigators: SAPS 3: from evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohort description. Intensive Care Med 2005, in press. Metnitz PhGH, Moreno RP, Almeida E, Jordan B, Bauer P, Abizanda-Campos R, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, on behalf of the SAPS 3 investigators: SAPS 3: from evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohort description. Intensive Care Med 2005, in press.
21.
go back to reference Moreno RP, Metnitz PhGH, Almeida E, Jordan B, Bauer P, Abizanda-Campos R, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, on behalf of the SAPS 3 investigators: SAPS 3: from evaluation of the patient to evaluation of the intensive care unit. Part 2: objectives, methods and cohort description. Intensive Care Med 2005, in press. Moreno RP, Metnitz PhGH, Almeida E, Jordan B, Bauer P, Abizanda-Campos R, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, on behalf of the SAPS 3 investigators: SAPS 3: from evaluation of the patient to evaluation of the intensive care unit. Part 2: objectives, methods and cohort description. Intensive Care Med 2005, in press.
22.
go back to reference Moine P, Hémery F, Blériot JP, Fulgencio JP, Garrigues B, Gouzes C, Le Gall JR, Lepage E, Villers D: [Completeness of ICU activity reports sent to French healthcare authorities]. Ann Fr Anesth Réanim 2004, 23: 15-20. 10.1016/j.annfar.2003.10.007CrossRefPubMed Moine P, Hémery F, Blériot JP, Fulgencio JP, Garrigues B, Gouzes C, Le Gall JR, Lepage E, Villers D: [Completeness of ICU activity reports sent to French healthcare authorities]. Ann Fr Anesth Réanim 2004, 23: 15-20. 10.1016/j.annfar.2003.10.007CrossRefPubMed
23.
go back to reference Holstein J, Taright N, Lepage E, Razafimamonjy J, Duboc D, Feldman L, Hittinger L, Lavergne T, Chatellier G: [Quality of medical database to valorize the DRG model by ISA cost indicators]. Rev Epidemiol Santé Publique 2002, 50: 593-603.PubMed Holstein J, Taright N, Lepage E, Razafimamonjy J, Duboc D, Feldman L, Hittinger L, Lavergne T, Chatellier G: [Quality of medical database to valorize the DRG model by ISA cost indicators]. Rev Epidemiol Santé Publique 2002, 50: 593-603.PubMed
Metadata
Title
Mortality prediction using SAPS II: an update for French intensive care units
Authors
Jean Roger Le Gall
Anke Neumann
François Hemery
Jean Pierre Bleriot
Jean Pierre Fulgencio
Bernard Garrigues
Christian Gouzes
Eric Lepage
Pierre Moine
Daniel Villers
Publication date
01-12-2005
Publisher
BioMed Central
Published in
Critical Care / Issue 6/2005
Electronic ISSN: 1364-8535
DOI
https://doi.org/10.1186/cc3821

Other articles of this Issue 6/2005

Critical Care 6/2005 Go to the issue