Skip to main content
Top
Published in: World Journal of Surgery 2/2018

01-02-2018 | Original Scientific Report

Derivation, Validation and Application of a Pragmatic Risk Prediction Index for Benchmarking of Surgical Outcomes

Authors: Richard T. Spence, David C. Chang, Haytham M. A. Kaafarani, Eugenio Panieri, Geoffrey A. Anderson, Matthew M. Hutter

Published in: World Journal of Surgery | Issue 2/2018

Login to get access

Abstract

Background

Despite the existence of multiple validated risk assessment and quality benchmarking tools in surgery, their utility outside of high-income countries is limited. We sought to derive, validate and apply a scoring system that is both (1) feasible, and (2) reliably predicts mortality in a middle-income country (MIC) context.

Methods

A 5-step methodology was used: (1) development of a de novo surgical outcomes database modeled around the American College of Surgeons’ National Surgical Quality Improvement Program (ACS-NSQIP) in South Africa (SA dataset), (2) use of the resultant data to identify all predictors of in-hospital death with more than 90% capture indicating feasibility of collection, (3) use these predictors to derive and validate an integer-based score that reliably predicts in-hospital death in the 2012 ACS-NSQIP, (4) apply the score in the original SA dataset and demonstrate its performance, (5) identify threshold cutoffs of the score to prompt action and drive quality improvement.

Results

Following step one-three above, the 13 point Codman’s score was derived and validated on 211,737 and 109,079 patients, respectively, and includes: age 65 (1), partially or completely dependent functional status (1), preoperative transfusions ≥4 units (1), emergency operation (2), sepsis or septic shock (2) American Society of Anesthesia score ≥3 (3) and operative procedure (1–3). Application of the score to 373 patients in the SA dataset showed good discrimination and calibration to predict an in-hospital death. A Codman Score of 8 is an optimal cutoff point for defining expected and unexpected deaths.

Conclusion

We have designed a novel risk prediction score specific for a MIC context. The Codman Score can prove useful for both (1) preoperative decision-making and (2) benchmarking the quality of surgical care in MIC’s.
Appendix
Available only for authorised users
Literature
1.
go back to reference Haider AH, Hashmi ZG, Gupta S, Zafar SN, David JS, Efron DT et al (2014) Benchmarking of trauma care worldwide: the potential value of an International Trauma Data Bank (ITDB). World J Surg 38(8):1882–1891. doi:10.1007/s00268-014-2629-5 CrossRefPubMed Haider AH, Hashmi ZG, Gupta S, Zafar SN, David JS, Efron DT et al (2014) Benchmarking of trauma care worldwide: the potential value of an International Trauma Data Bank (ITDB). World J Surg 38(8):1882–1891. doi:10.​1007/​s00268-014-2629-5 CrossRefPubMed
2.
go back to reference Pearse RM, Moreno RP, Bauer P, Pelosi P, Metnitz P, Spies C et al (2012) Mortality after surgery in Europe: a 7 day cohort study. Lancet 380(9847):1059–1065CrossRefPubMedPubMedCentral Pearse RM, Moreno RP, Bauer P, Pelosi P, Metnitz P, Spies C et al (2012) Mortality after surgery in Europe: a 7 day cohort study. Lancet 380(9847):1059–1065CrossRefPubMedPubMedCentral
3.
go back to reference Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP et al (2009) A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 360(5):491–499CrossRefPubMed Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP et al (2009) A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 360(5):491–499CrossRefPubMed
4.
go back to reference Meara JG, Greenberg SL (2015) Global surgery as an equal partner in health: no longer the neglected stepchild. Lancet Glob Health 27(3 Suppl 2):S1–S2CrossRef Meara JG, Greenberg SL (2015) Global surgery as an equal partner in health: no longer the neglected stepchild. Lancet Glob Health 27(3 Suppl 2):S1–S2CrossRef
5.
go back to reference Farmer DL (2012) NSQIP lite: a potential tool for global comparative effectiveness evaluations. Arch Surg 147(9):803–804CrossRefPubMed Farmer DL (2012) NSQIP lite: a potential tool for global comparative effectiveness evaluations. Arch Surg 147(9):803–804CrossRefPubMed
6.
go back to reference Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY (2009) Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 250(3):363–376PubMed Hall BL, Hamilton BH, Richards K, Bilimoria KY, Cohen ME, Ko CY (2009) Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 250(3):363–376PubMed
7.
go back to reference Cohen ME, Liu Y, Ko CY, Hall BL (2015) Improved surgical outcomes for ACS NSQIP hospitals over time: evaluation of hospital cohorts with up to 8 years of participation. Ann Surg 263(2):267–273CrossRef Cohen ME, Liu Y, Ko CY, Hall BL (2015) Improved surgical outcomes for ACS NSQIP hospitals over time: evaluation of hospital cohorts with up to 8 years of participation. Ann Surg 263(2):267–273CrossRef
8.
go back to reference Haynes AB, Regenbogen SE, Weiser TG, Lipsitz SR, Dziekan G, Berry WR et al (2011) Surgical outcome measurement for a global patient population: validation of the Surgical Apgar Score in 8 countries. Surgery 149(4):519–524CrossRefPubMed Haynes AB, Regenbogen SE, Weiser TG, Lipsitz SR, Dziekan G, Berry WR et al (2011) Surgical outcome measurement for a global patient population: validation of the Surgical Apgar Score in 8 countries. Surgery 149(4):519–524CrossRefPubMed
9.
go back to reference Berwick DM (2015) Measuring surgical outcomes for improvement: was Codman wrong? JAMA 313(5):469–470CrossRefPubMed Berwick DM (2015) Measuring surgical outcomes for improvement: was Codman wrong? JAMA 313(5):469–470CrossRefPubMed
11.
go back to reference Johnson RG, Arozullah AM, Neumayer L, Henderson WG, Hosokawa P, Khuri SF (2007) Multivariable predictors of postoperative respiratory failure after general and vascular surgery: results from the patient safety in surgery study. J Am Coll Surg 204(6):1188–1198CrossRefPubMed Johnson RG, Arozullah AM, Neumayer L, Henderson WG, Hosokawa P, Khuri SF (2007) Multivariable predictors of postoperative respiratory failure after general and vascular surgery: results from the patient safety in surgery study. J Am Coll Surg 204(6):1188–1198CrossRefPubMed
12.
go back to reference Andersen J, Lassiter R, Bickler S, Talamini M, Chang D (2012) Brief tool to measure risk-adjusted surgical outcomes in resource-limited hospitals. Arch Surg 147(9):798–803CrossRef Andersen J, Lassiter R, Bickler S, Talamini M, Chang D (2012) Brief tool to measure risk-adjusted surgical outcomes in resource-limited hospitals. Arch Surg 147(9):798–803CrossRef
14.
go back to reference Lee TH, Marcantonio ER, Mangione CM, Thomas EJ, Polanczyk CA, Cook EF et al (1999) Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation 100(10):1043–1049CrossRefPubMed Lee TH, Marcantonio ER, Mangione CM, Thomas EJ, Polanczyk CA, Cook EF et al (1999) Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation 100(10):1043–1049CrossRefPubMed
15.
go back to reference Batterton KA, Schubert CM (2015) A nonparametric fiducial interval for the Youden index in multi-state diagnostic settings. Stat Med 35(1):78–96CrossRefPubMed Batterton KA, Schubert CM (2015) A nonparametric fiducial interval for the Youden index in multi-state diagnostic settings. Stat Med 35(1):78–96CrossRefPubMed
16.
go back to reference Rubinfeld I, Farooq M, Velanovich V, Syed Z (2010) Predicting surgical risk: how much data is enough? AMIA Annu Symp Proc. 2010:777–781PubMedPubMedCentral Rubinfeld I, Farooq M, Velanovich V, Syed Z (2010) Predicting surgical risk: how much data is enough? AMIA Annu Symp Proc. 2010:777–781PubMedPubMedCentral
17.
go back to reference Dimick JB, Osborne NH, Hall BL, Ko CY, Birkmeyer JD (2010) Risk adjustment for comparing hospital quality with surgery: how many variables are needed? J Am Coll Surg 210(4):503–508CrossRefPubMedPubMedCentral Dimick JB, Osborne NH, Hall BL, Ko CY, Birkmeyer JD (2010) Risk adjustment for comparing hospital quality with surgery: how many variables are needed? J Am Coll Surg 210(4):503–508CrossRefPubMedPubMedCentral
18.
go back to reference Ariyaratnam R, Palmqvist CL, Hider P, Laing GL, Stupart D, Wilson L et al (2015) Toward a standard approach to measurement and reporting of perioperative mortality rate as a global indicator for surgery. Surgery 158(1):17–26CrossRefPubMed Ariyaratnam R, Palmqvist CL, Hider P, Laing GL, Stupart D, Wilson L et al (2015) Toward a standard approach to measurement and reporting of perioperative mortality rate as a global indicator for surgery. Surgery 158(1):17–26CrossRefPubMed
19.
go back to reference Henderson WG, Daley J (2009) Design and statistical methodology of the National Surgical Quality Improvement Program: why is it what it is? Am J Surg 198(5 Suppl):S19–S27CrossRefPubMed Henderson WG, Daley J (2009) Design and statistical methodology of the National Surgical Quality Improvement Program: why is it what it is? Am J Surg 198(5 Suppl):S19–S27CrossRefPubMed
20.
go back to reference Haynes SR, Lawler PG (1995) An assessment of the consistency of ASA physical status classification allocation. Anaesthesia 50(3):195–199CrossRefPubMed Haynes SR, Lawler PG (1995) An assessment of the consistency of ASA physical status classification allocation. Anaesthesia 50(3):195–199CrossRefPubMed
21.
go back to reference Cohen ME, Bilimoria KY, Ko CY, Richards K, Hall BL (2009) Effect of subjective preoperative variables on risk-adjusted assessment of hospital morbidity and mortality. Ann Surg 249(4):682–689CrossRefPubMed Cohen ME, Bilimoria KY, Ko CY, Richards K, Hall BL (2009) Effect of subjective preoperative variables on risk-adjusted assessment of hospital morbidity and mortality. Ann Surg 249(4):682–689CrossRefPubMed
Metadata
Title
Derivation, Validation and Application of a Pragmatic Risk Prediction Index for Benchmarking of Surgical Outcomes
Authors
Richard T. Spence
David C. Chang
Haytham M. A. Kaafarani
Eugenio Panieri
Geoffrey A. Anderson
Matthew M. Hutter
Publication date
01-02-2018
Publisher
Springer International Publishing
Published in
World Journal of Surgery / Issue 2/2018
Print ISSN: 0364-2313
Electronic ISSN: 1432-2323
DOI
https://doi.org/10.1007/s00268-017-4177-2

Other articles of this Issue 2/2018

World Journal of Surgery 2/2018 Go to the issue