Skip to main content
Top
Published in: Intensive Care Medicine 1/2010

01-01-2010 | Pediatric Original

External validation of the paediatric logistic organ dysfunction score

Authors: Pedro Celiny R. Garcia, Pablo Eulmesekian, Ricardo G. Branco, Augusto Perez, Ana Sffogia, Lorenzo Olivero, Jefferson P. Piva, Robert C. Tasker

Published in: Intensive Care Medicine | Issue 1/2010

Login to get access

Abstract

Objective

External validation of the paediatric logistic organ dysfunction (PELOD) score in two paediatric intensive care units (PICU) in South America.

Methods

Prospective observational cohort study including all PICU admissions from July 2003 to December 2004 in Porto Alegre, Brazil, and from January 2004 to December 2004 in Buenos Aires, Argentina. The data collected included demographic variables, diagnosis, need for mechanical ventilation, length of PICU stay and mortality, and the 12 variables in the PELOD score. For each PELOD score variable, the worst daily value and the worst value of the whole PICU stay were used for the daily PELOD (dPELOD) and PELOD scores, respectively.

Results

A total of 1,476 admissions (51.3% from Argentina and 48.7% from Brazil) were analysed. Observed and predicted mortality were, respectively, 4.7% and 6.6%, with a standardized mortality ratio of 0.72. The score showed excellent discrimination capacity, with an area under the receiver operator characteristic (ROC) curve of 0.93 (0.88–0.98). The dPELOD score on days 1–5 also showed good discrimination capacities, with areas under the ROC curve >0.85. However, PELOD and dPELOD scores showed poor calibration with the Hosmer–Lemeshow test (chi-square 72.3, p < 0.001). This poor calibration was explained by a deficiency in the PELOD score where it fails to identify two risk intervals; 3.1–16.2% and 40–80%.

Conclusions

The PELOD score is reproducible, has excellent discrimination, but over-predicts mortality and has poor calibration. Although the lack of calibration may not invalidate the score, the PELOD score is a discontinuous variable and we advise careful consideration when using it as a surrogate endpoint in clinical trials.
Literature
1.
go back to reference Leteurtre S, Martinot A, Duhamel A, Gauvin F, Grandbastien B, Nam TV, Proulx F, Lacroix J, Leclerc F (1999) Development of a pediatric multiple organ dysfunction score: use of two strategies. Med Decis Making 19:399–410CrossRefPubMed Leteurtre S, Martinot A, Duhamel A, Gauvin F, Grandbastien B, Nam TV, Proulx F, Lacroix J, Leclerc F (1999) Development of a pediatric multiple organ dysfunction score: use of two strategies. Med Decis Making 19:399–410CrossRefPubMed
2.
go back to reference Leteurtre S, Martinot A, Duhamel A, Proulx F, Grandbastien B, Cotting J, Gottesman R, Joffe A, Pfenninger J, Hubert P, Lacroix J, Leclerc F (2003) Validation of the paediatric logistic organ dysfunction (PELOD) score: prospective, observational, multicentre study. Lancet 362:192–197CrossRefPubMed Leteurtre S, Martinot A, Duhamel A, Proulx F, Grandbastien B, Cotting J, Gottesman R, Joffe A, Pfenninger J, Hubert P, Lacroix J, Leclerc F (2003) Validation of the paediatric logistic organ dysfunction (PELOD) score: prospective, observational, multicentre study. Lancet 362:192–197CrossRefPubMed
3.
go back to reference Lacroix J, Cotting J (2005) Severity of illness and organ dysfunction scoring in children. Pediatr Crit Care Med 6:S126–S134CrossRefPubMed Lacroix J, Cotting J (2005) Severity of illness and organ dysfunction scoring in children. Pediatr Crit Care Med 6:S126–S134CrossRefPubMed
4.
go back to reference Leteurtre S, Duhamel A, Grandbastien B, Lacroix J, Leclerc F (2006) Paediatric logistic organ dysfunction (PELOD) score. Lancet 367:897 author reply 900–892CrossRefPubMed Leteurtre S, Duhamel A, Grandbastien B, Lacroix J, Leclerc F (2006) Paediatric logistic organ dysfunction (PELOD) score. Lancet 367:897 author reply 900–892CrossRefPubMed
5.
go back to reference Leteurtre S, Duhamel A, Grandbastien B, Lacroix J, Leclerc F (2006) Paediatric logistic organ dysfunction (PELOD) score. Lancet 367:902CrossRef Leteurtre S, Duhamel A, Grandbastien B, Lacroix J, Leclerc F (2006) Paediatric logistic organ dysfunction (PELOD) score. Lancet 367:902CrossRef
9.
go back to reference Timsit JF, Fosse JP, Troche G, De Lassence A, Alberti C, Garrouste-Orgeas M, Bornstain C, Adrie C, Cheval C, Chevret S (2002) Calibration and discrimination by daily logistic organ dysfunction scoring comparatively with daily sequential organ failure assessment scoring for predicting hospital mortality in critically ill patients. Crit Care Med 30:2003–2013CrossRefPubMed Timsit JF, Fosse JP, Troche G, De Lassence A, Alberti C, Garrouste-Orgeas M, Bornstain C, Adrie C, Cheval C, Chevret S (2002) Calibration and discrimination by daily logistic organ dysfunction scoring comparatively with daily sequential organ failure assessment scoring for predicting hospital mortality in critically ill patients. Crit Care Med 30:2003–2013CrossRefPubMed
10.
go back to reference Pollack MM, Ruttimann UE, Getson PR (1988) Pediatric risk of mortality (PRISM) score. Crit Care Med 16:1110–1116PubMedCrossRef Pollack MM, Ruttimann UE, Getson PR (1988) Pediatric risk of mortality (PRISM) score. Crit Care Med 16:1110–1116PubMedCrossRef
11.
go back to reference Balakrishnan G, Aitchison T, Hallworth D, Morton NS (1992) Prospective evaluation of the paediatric risk of mortality (PRISM) score. Arch Dis Child 67:196–200CrossRefPubMed Balakrishnan G, Aitchison T, Hallworth D, Morton NS (1992) Prospective evaluation of the paediatric risk of mortality (PRISM) score. Arch Dis Child 67:196–200CrossRefPubMed
12.
go back to reference Slater A, Shann F, Pearson G (2003) PIM2: a revised version of the paediatric index of mortality. Intensive Care Med 29:278–285PubMed Slater A, Shann F, Pearson G (2003) PIM2: a revised version of the paediatric index of mortality. Intensive Care Med 29:278–285PubMed
13.
go back to reference Martha VF, Garcia PC, Piva JP, Einloft PR, Bruno F, Rampon V (2005) Comparison of two prognostic scores (PRISM and PIM) at a padiatric intensive care unit. Jornal de pediatria 81:259–264PubMed Martha VF, Garcia PC, Piva JP, Einloft PR, Bruno F, Rampon V (2005) Comparison of two prognostic scores (PRISM and PIM) at a padiatric intensive care unit. Jornal de pediatria 81:259–264PubMed
14.
go back to reference Eulmesekian PG, Perez A, Minces PG, Ferrero H (2007) Validation of pediatric index of mortality 2 (PIM2) in a single pediatric intensive care unit of Argentina. Pediatr Crit Care Med 8:54–57CrossRefPubMed Eulmesekian PG, Perez A, Minces PG, Ferrero H (2007) Validation of pediatric index of mortality 2 (PIM2) in a single pediatric intensive care unit of Argentina. Pediatr Crit Care Med 8:54–57CrossRefPubMed
15.
go back to reference Bleeker SE, Moll HA, Steyerberg EW, Donders AR, Derksen-Lubsen G, Grobbee DE, Moons KG (2003) External validation is necessary in prediction research: a clinical example. J Clin Epidemiol 56:826–832CrossRefPubMed Bleeker SE, Moll HA, Steyerberg EW, Donders AR, Derksen-Lubsen G, Grobbee DE, Moons KG (2003) External validation is necessary in prediction research: a clinical example. J Clin Epidemiol 56:826–832CrossRefPubMed
16.
go back to reference Terrin N, Schmid CH, Griffith JL, D’Agostino RB, Selker HP (2003) External validity of predictive models: a comparison of logistic regression, classification trees, and neural networks. J Clin Epidemiol 56:721–729CrossRefPubMed Terrin N, Schmid CH, Griffith JL, D’Agostino RB, Selker HP (2003) External validity of predictive models: a comparison of logistic regression, classification trees, and neural networks. J Clin Epidemiol 56:721–729CrossRefPubMed
17.
go back to reference Graciano AL, Balko JA, Rahn DS, Ahmad N, Giroir BP (2005) The pediatric multiple organ dysfunction score (P-MODS): development and validation of an objective scale to measure the severity of multiple organ dysfunction in critically ill children. Crit Care Med 33:1484–1491CrossRefPubMed Graciano AL, Balko JA, Rahn DS, Ahmad N, Giroir BP (2005) The pediatric multiple organ dysfunction score (P-MODS): development and validation of an objective scale to measure the severity of multiple organ dysfunction in critically ill children. Crit Care Med 33:1484–1491CrossRefPubMed
18.
go back to reference Flora JD (1978) A method for comparing survival of burn patients to a standard survival curve. J Trauma 18:701–708CrossRefPubMed Flora JD (1978) A method for comparing survival of burn patients to a standard survival curve. J Trauma 18:701–708CrossRefPubMed
19.
go back to reference Garcia PC, Eulmesekian P, Sffogia A, Perez A, Branco RG, Piva JP, Tasker RC (2006) Limitation in paediatric logistic organ dysfunction score. Lancet 368:1151 author reply 1151–1152CrossRefPubMed Garcia PC, Eulmesekian P, Sffogia A, Perez A, Branco RG, Piva JP, Tasker RC (2006) Limitation in paediatric logistic organ dysfunction score. Lancet 368:1151 author reply 1151–1152CrossRefPubMed
20.
go back to reference Khemani RG, Markivitz BP, Curley MAQ (2007) Epidemiologic factors of mechanically ventilated PICU patients in the United States. Pediatr Crit Care Med: A39 Khemani RG, Markivitz BP, Curley MAQ (2007) Epidemiologic factors of mechanically ventilated PICU patients in the United States. Pediatr Crit Care Med: A39
21.
go back to reference Randolph AG, Meert KL, O’Neil ME, Hanson JH, Luckett PM, Arnold JH, Gedeit RG, Cox PN, Roberts JS, Venkataraman ST, Forbes PW, Cheifetz IM (2003) The feasibility of conducting clinical trials in infants and children with acute respiratory failure. Am J Respir Crit Care Med 167:1334–1340CrossRefPubMed Randolph AG, Meert KL, O’Neil ME, Hanson JH, Luckett PM, Arnold JH, Gedeit RG, Cox PN, Roberts JS, Venkataraman ST, Forbes PW, Cheifetz IM (2003) The feasibility of conducting clinical trials in infants and children with acute respiratory failure. Am J Respir Crit Care Med 167:1334–1340CrossRefPubMed
22.
go back to reference Ream RS, Mackey K, Leet T, Green MC, Andreone TL, Loftis LL, Lynch RE (2007) Association of nursing workload and unplanned extubations in a pediatric intensive care unit. Pediatr Crit Care Med 8:366–371CrossRefPubMed Ream RS, Mackey K, Leet T, Green MC, Andreone TL, Loftis LL, Lynch RE (2007) Association of nursing workload and unplanned extubations in a pediatric intensive care unit. Pediatr Crit Care Med 8:366–371CrossRefPubMed
23.
go back to reference Peek N, Arts DG, Bosman RJ, van der Voort PH, de Keizer NF (2007) External validation of prognostic models for critically ill patients required substantial sample sizes. J Clin Epidemiol 60:491–501CrossRefPubMed Peek N, Arts DG, Bosman RJ, van der Voort PH, de Keizer NF (2007) External validation of prognostic models for critically ill patients required substantial sample sizes. J Clin Epidemiol 60:491–501CrossRefPubMed
24.
go back to reference Marcin JP, Pollack MM (2000) Review of the methodologies and applications of scoring systems in neonatal and pediatric intensive care. Pediatr Crit Care Med 1:20–27CrossRefPubMed Marcin JP, Pollack MM (2000) Review of the methodologies and applications of scoring systems in neonatal and pediatric intensive care. Pediatr Crit Care Med 1:20–27CrossRefPubMed
25.
go back to reference Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19:453–473CrossRefPubMed Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19:453–473CrossRefPubMed
26.
go back to reference Pollack MM (2003) Prognostication scores. In: Field MJ, Behrman RE (eds) When children die: improving palliative care and end-of-life care for children and their families. The National Academies Press, Washington DC, pp 449–475 Pollack MM (2003) Prognostication scores. In: Field MJ, Behrman RE (eds) When children die: improving palliative care and end-of-life care for children and their families. The National Academies Press, Washington DC, pp 449–475
27.
go back to reference Thukral A, Kohli U, Lodha R, Kabra SK, Kabra NK (2007) Validation of the PELOD score for multiple organ dysfunction in children. Indian Pediatr 44:683–686PubMed Thukral A, Kohli U, Lodha R, Kabra SK, Kabra NK (2007) Validation of the PELOD score for multiple organ dysfunction in children. Indian Pediatr 44:683–686PubMed
28.
go back to reference Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S (1997) A comparison of goodness-of-fit tests for the logistic regression model. Stat Med 16:965–980CrossRefPubMed Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S (1997) A comparison of goodness-of-fit tests for the logistic regression model. Stat Med 16:965–980CrossRefPubMed
29.
go back to reference Leteurtre S, Duhamel A, Grandbastien B, Lacroix J, Leclerc F (2006) Paediatric logistic organ dysfunction (PELOD) score. Lancet 368:1151–1152CrossRef Leteurtre S, Duhamel A, Grandbastien B, Lacroix J, Leclerc F (2006) Paediatric logistic organ dysfunction (PELOD) score. Lancet 368:1151–1152CrossRef
Metadata
Title
External validation of the paediatric logistic organ dysfunction score
Authors
Pedro Celiny R. Garcia
Pablo Eulmesekian
Ricardo G. Branco
Augusto Perez
Ana Sffogia
Lorenzo Olivero
Jefferson P. Piva
Robert C. Tasker
Publication date
01-01-2010
Publisher
Springer-Verlag
Published in
Intensive Care Medicine / Issue 1/2010
Print ISSN: 0342-4642
Electronic ISSN: 1432-1238
DOI
https://doi.org/10.1007/s00134-009-1489-1

Other articles of this Issue 1/2010

Intensive Care Medicine 1/2010 Go to the issue