Skip to main content
Top
Published in: Critical Care 1/2018

Open Access 01-12-2018 | Research

Development and internal validation of the multivariable CIPHER (Collaborative Integrated Pregnancy High-dependency Estimate of Risk) clinical risk prediction model

Authors: Beth A. Payne, Helen Ryan, Jeffrey Bone, Laura A. Magee, Alice B. Aarvold, J. Mark Ansermino, Zulfiqar A. Bhutta, Mary Bowen, J. Guilherme Cecatti, Cynthia Chazotte, Tim Crozier, Anne-Cornélie J. M. de Pont, Oktay Demirkiran, Tao Duan, Marlot Kallen, Wessel Ganzevoort, Michael Geary, Dena Goffman, Jennifer A. Hutcheon, K. S. Joseph, Stephen E. Lapinsky, Isam Lataifeh, Jing Li, Sarka Liskonova, Emily M. Hamel, Fionnuala M. McAuliffe, Colm O’Herlihy, Ben W. J. Mol, P. Gareth R. Seaward, Ramzy Tadros, Turkan Togal, Rahat Qureshi, U. Vivian Ukah, Daniela Vasquez, Euan Wallace, Paul Yong, Vivian Zhou, Keith R. Walley, Peter von Dadelszen, the CIPHER Group

Published in: Critical Care | Issue 1/2018

Login to get access

Abstract

Background

Intensive care unit (ICU) outcome prediction models, such as Acute Physiology And Chronic Health Evaluation (APACHE), were designed in general critical care populations and their use in obstetric populations is contentious. The aim of the CIPHER (Collaborative Integrated Pregnancy High-dependency Estimate of Risk) study was to develop and internally validate a multivariable prognostic model calibrated specifically for pregnant or recently delivered women admitted for critical care.

Methods

A retrospective observational cohort was created for this study from 13 tertiary facilities across five high-income and six low- or middle-income countries. Women admitted to an ICU for more than 24 h during pregnancy or less than 6 weeks post-partum from 2000 to 2012 were included in the cohort. A composite primary outcome was defined as maternal death or need for organ support for more than 7 days or acute life-saving intervention. Model development involved selection of candidate predictor variables based on prior evidence of effect, availability across study sites, and use of LASSO (Least Absolute Shrinkage and Selection Operator) model building after multiple imputation using chained equations to address missing data for variable selection. The final model was estimated using multivariable logistic regression. Internal validation was completed using bootstrapping to correct for optimism in model performance measures of discrimination and calibration.

Results

Overall, 127 out of 769 (16.5%) women experienced an adverse outcome. Predictors included in the final CIPHER model were maternal age, surgery in the preceding 24 h, systolic blood pressure, Glasgow Coma Scale score, serum sodium, serum potassium, activated partial thromboplastin time, arterial blood gas (ABG) pH, serum creatinine, and serum bilirubin. After internal validation, the model maintained excellent discrimination (area under the curve of the receiver operating characteristic (AUROC) 0.82, 95% confidence interval (CI) 0.81 to 0.84) and good calibration (slope of 0.92, 95% CI 0.91 to 0.92 and intercept of −0.11, 95% CI −0.13 to −0.08).

Conclusions

The CIPHER model has the potential to be a pragmatic risk prediction tool. CIPHER can identify critically ill pregnant women at highest risk for adverse outcomes, inform counseling of patients about risk, and facilitate bench-marking of outcomes between centers by adjusting for baseline risk.
Appendix
Available only for authorised users
Literature
1.
go back to reference Ryan HM, Sharma S, Magee LA, Ansermino JM, MacDonell K, Payne BA, et al. The Usefulness of the APACHE II Score in Obstetric Critical Care: A Structured Review. J Obstet Gynaecol Can. 2016;38:909–18.CrossRef Ryan HM, Sharma S, Magee LA, Ansermino JM, MacDonell K, Payne BA, et al. The Usefulness of the APACHE II Score in Obstetric Critical Care: A Structured Review. J Obstet Gynaecol Can. 2016;38:909–18.CrossRef
2.
go back to reference Crozier TM, Wallace EM. Obstetric admissions to an integrated general intensive care unit in a quaternary maternity facility. Aust N Z J Obstet Gynaecol. 2011;51:233–8.CrossRef Crozier TM, Wallace EM. Obstetric admissions to an integrated general intensive care unit in a quaternary maternity facility. Aust N Z J Obstet Gynaecol. 2011;51:233–8.CrossRef
3.
go back to reference Lapinsky SE, Kruczynski K, Seaward GR, Farine D, Grossman RF. Critical care management of the obstetric patient. Can J Anaesth. 1997;44:325–9.CrossRef Lapinsky SE, Kruczynski K, Seaward GR, Farine D, Grossman RF. Critical care management of the obstetric patient. Can J Anaesth. 1997;44:325–9.CrossRef
4.
go back to reference Cook R, Cook D, Tilley J, Lee K, Marshall J. Multiple organ dysfunction: baseline and serial component scores. Crit Care Med. 2001;29:2046–50.CrossRef Cook R, Cook D, Tilley J, Lee K, Marshall J. Multiple organ dysfunction: baseline and serial component scores. Crit Care Med. 2001;29:2046–50.CrossRef
5.
go back to reference Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13:818–29.CrossRef Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13:818–29.CrossRef
6.
go back to reference Metnitz PG, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med. 2005;31:1336–44.CrossRef Metnitz PG, Moreno RP, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. Intensive Care Med. 2005;31:1336–44.CrossRef
7.
go back to reference Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31:1345–55.CrossRef Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31:1345–55.CrossRef
8.
go back to reference Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707–10.CrossRef Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707–10.CrossRef
9.
go back to reference Oliveira-Neto A, Parpinelli MA, Cecatti JG, Souza JP, Sousa MH. Sequential organ failure assessment score for evaluating organ failure and outcome of severe maternal morbidity in obstetric intensive care. ScientificWorldJournal. 2012;2012:172145.CrossRef Oliveira-Neto A, Parpinelli MA, Cecatti JG, Souza JP, Sousa MH. Sequential organ failure assessment score for evaluating organ failure and outcome of severe maternal morbidity in obstetric intensive care. ScientificWorldJournal. 2012;2012:172145.CrossRef
10.
go back to reference Souza JP, Cecatti JG, Haddad SM, Parpinelli MA, Costa ML, Katz L, et al. The WHO maternal near-miss approach and the maternal severity index model (MSI): tools for assessing the management of severe maternal morbidity. PLoS One. 2012;7:e44129.CrossRef Souza JP, Cecatti JG, Haddad SM, Parpinelli MA, Costa ML, Katz L, et al. The WHO maternal near-miss approach and the maternal severity index model (MSI): tools for assessing the management of severe maternal morbidity. PLoS One. 2012;7:e44129.CrossRef
11.
go back to reference Ryan H, Jones M, Payne B, Sharma S, Hutfield AM, Lee T, et al. Validating the performance of the Modified Early Obstetric Warning System multivariable model to predict maternal intensive care admission. J Obstet Gynaecol Can. 2017;39:728–33.CrossRef Ryan H, Jones M, Payne B, Sharma S, Hutfield AM, Lee T, et al. Validating the performance of the Modified Early Obstetric Warning System multivariable model to predict maternal intensive care admission. J Obstet Gynaecol Can. 2017;39:728–33.CrossRef
12.
go back to reference Lockitch G. Handbook of diagnostic biochemistry and haematology in normal pregnancy. Boca Raton: CRC Press; 1993. Lockitch G. Handbook of diagnostic biochemistry and haematology in normal pregnancy. Boca Raton: CRC Press; 1993.
13.
go back to reference Philipp EE, Barnes J, Newton M. Scientific foundations of obstetrics and gynaecology. 3rd ed. Oxford: Butterworth-Heinemann Ltd; 1987. Philipp EE, Barnes J, Newton M. Scientific foundations of obstetrics and gynaecology. 3rd ed. Oxford: Butterworth-Heinemann Ltd; 1987.
14.
go back to reference Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49:1373–9.CrossRef Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49:1373–9.CrossRef
15.
go back to reference Lapinsky SE, Hallett D, Collop N, Drover J, Lavercombe P, Leeman M, et al. Evaluation of standard and modified severity of illness scores in the obstetric patient. J Crit Care. 2011;26:535–7.CrossRef Lapinsky SE, Hallett D, Collop N, Drover J, Lavercombe P, Leeman M, et al. Evaluation of standard and modified severity of illness scores in the obstetric patient. J Crit Care. 2011;26:535–7.CrossRef
17.
go back to reference Donders ART, van der Heijden GJMG, Stijnen T, Moons KGM. Review: A gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59:1087–91.CrossRef Donders ART, van der Heijden GJMG, Stijnen T, Moons KGM. Review: A gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59:1087–91.CrossRef
18.
go back to reference Moons KGM, Donders RART, Stijnen T, Harrell FE Jr. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol. 2006;59:1092–101.CrossRef Moons KGM, Donders RART, Stijnen T, Harrell FE Jr. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol. 2006;59:1092–101.CrossRef
19.
go back to reference Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc. 1996;58:267–88. Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc. 1996;58:267–88.
21.
go back to reference Rubin D. Multiple Imputation for Nonresponse in Surveys. New York: Wiley; 1987. Rubin D. Multiple Imputation for Nonresponse in Surveys. New York: Wiley; 1987.
22.
go back to reference Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36.CrossRef Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36.CrossRef
23.
go back to reference De Cock B, Nieboer D, Van Calster B, Steyerberg E, Vergouwe Y (2016). CalibrationCurves:Calibration performance. R package version 0.1.2. De Cock B, Nieboer D, Van Calster B, Steyerberg E, Vergouwe Y (2016). CalibrationCurves:Calibration performance. R package version 0.1.2.
25.
go back to reference Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–38.CrossRef Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–38.CrossRef
26.
go back to reference Deeks J, Altman D. Statistics notes - Diagnostic tests 4: likelihood ratios. Br Med J. 2004;329:168–9.CrossRef Deeks J, Altman D. Statistics notes - Diagnostic tests 4: likelihood ratios. Br Med J. 2004;329:168–9.CrossRef
28.
go back to reference Payne BA, Hutcheon JA, Ansermino JM, Hall DR, Bhutta ZA, Bhutta SZ, et al. A risk prediction model for the assessment and triage of women with hypertensive disorders of pregnancy in low-resourced settings: the miniPIERS (Pre-eclampsia Integrated Estimate of RiSk) multi-country prospective cohort study. PLoS Med. 2014;11:e1001589.CrossRef Payne BA, Hutcheon JA, Ansermino JM, Hall DR, Bhutta ZA, Bhutta SZ, et al. A risk prediction model for the assessment and triage of women with hypertensive disorders of pregnancy in low-resourced settings: the miniPIERS (Pre-eclampsia Integrated Estimate of RiSk) multi-country prospective cohort study. PLoS Med. 2014;11:e1001589.CrossRef
29.
go back to reference von Dadelszen P, Payne B, Li J, Ansermino JM, Broughton PF, Cote AM, et al. Prediction of adverse maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model. Lancet. 2011;377:219–27.CrossRef von Dadelszen P, Payne B, Li J, Ansermino JM, Broughton PF, Cote AM, et al. Prediction of adverse maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model. Lancet. 2011;377:219–27.CrossRef
30.
go back to reference Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.CrossRef Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.CrossRef
31.
go back to reference Harrison DA, Penny JA, Yentis SM, Fayek S, Brady AR. Case mix, outcome and activity for obstetric admissions to adult, general critical care units: a secondary analysis of the ICNARC Case Mix Programme Database. Crit Care. 2005;9(Suppl 3):S25–37.CrossRef Harrison DA, Penny JA, Yentis SM, Fayek S, Brady AR. Case mix, outcome and activity for obstetric admissions to adult, general critical care units: a secondary analysis of the ICNARC Case Mix Programme Database. Crit Care. 2005;9(Suppl 3):S25–37.CrossRef
32.
go back to reference Huchon C, Dumont A, Traore M, Abrahamowicz M, Fauconnier A, Fraser W, et al. A prediction score for maternal mortality in Senegal and Mali. Obstet Gynecol. 2013;121:1049–56.CrossRef Huchon C, Dumont A, Traore M, Abrahamowicz M, Fauconnier A, Fraser W, et al. A prediction score for maternal mortality in Senegal and Mali. Obstet Gynecol. 2013;121:1049–56.CrossRef
33.
go back to reference Novicoff WM, Wagner DP, Knaus WA, Kane EK, Cecere F, Draper E, et al. Initial development of a system-wide maternal-fetal outcomes assessment program. Am J Obstet Gynecol. 2000;183:291–300.CrossRef Novicoff WM, Wagner DP, Knaus WA, Kane EK, Cecere F, Draper E, et al. Initial development of a system-wide maternal-fetal outcomes assessment program. Am J Obstet Gynecol. 2000;183:291–300.CrossRef
34.
go back to reference Staff AC, Burke O, Benton S, von Dadelszen P, Szafranski P, Zhang C, et al. Maternal circulating PlGF concentrations and placenta-related pregnancy complications: First results from the CoLab AngF Study. Pregnancy Hypertens. 2013;3:59.CrossRef Staff AC, Burke O, Benton S, von Dadelszen P, Szafranski P, Zhang C, et al. Maternal circulating PlGF concentrations and placenta-related pregnancy complications: First results from the CoLab AngF Study. Pregnancy Hypertens. 2013;3:59.CrossRef
35.
go back to reference Goldberg SA, Kharbanda B, Pepe PE. Year in review 2013: Critical Care--out-of-hospital cardiac arrest, traumatic injury, and other emergency care conditions. Crit Care. 2014;18:593.CrossRef Goldberg SA, Kharbanda B, Pepe PE. Year in review 2013: Critical Care--out-of-hospital cardiac arrest, traumatic injury, and other emergency care conditions. Crit Care. 2014;18:593.CrossRef
Metadata
Title
Development and internal validation of the multivariable CIPHER (Collaborative Integrated Pregnancy High-dependency Estimate of Risk) clinical risk prediction model
Authors
Beth A. Payne
Helen Ryan
Jeffrey Bone
Laura A. Magee
Alice B. Aarvold
J. Mark Ansermino
Zulfiqar A. Bhutta
Mary Bowen
J. Guilherme Cecatti
Cynthia Chazotte
Tim Crozier
Anne-Cornélie J. M. de Pont
Oktay Demirkiran
Tao Duan
Marlot Kallen
Wessel Ganzevoort
Michael Geary
Dena Goffman
Jennifer A. Hutcheon
K. S. Joseph
Stephen E. Lapinsky
Isam Lataifeh
Jing Li
Sarka Liskonova
Emily M. Hamel
Fionnuala M. McAuliffe
Colm O’Herlihy
Ben W. J. Mol
P. Gareth R. Seaward
Ramzy Tadros
Turkan Togal
Rahat Qureshi
U. Vivian Ukah
Daniela Vasquez
Euan Wallace
Paul Yong
Vivian Zhou
Keith R. Walley
Peter von Dadelszen
the CIPHER Group
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Critical Care / Issue 1/2018
Electronic ISSN: 1364-8535
DOI
https://doi.org/10.1186/s13054-018-2215-6

Other articles of this Issue 1/2018

Critical Care 1/2018 Go to the issue