Skip to main content
Top
Published in: Journal of Gastrointestinal Surgery 5/2014

01-05-2014 | Original Article

Development of an Improved Risk Calculator for Complications in Proctectomy

Authors: Scott K. Sherman, Jennifer E. Hrabe, Mary E. Charlton, John W. Cromwell, John C. Byrn

Published in: Journal of Gastrointestinal Surgery | Issue 5/2014

Login to get access

Abstract

Background

Rectal surgery is associated with high complication rates, but tools to prospectively define surgical risk are lacking. Improved preoperative risk assessment could better inform patients and refine decision making by surgeons. Our objective was to develop a validated model for proctectomy risk prediction.

Methods

We reviewed non-emergent ACS-NSQIP proctectomy data from 2005 to 2011 (n = 13,385). Logistic regression identified variables available prior to surgery showing independent association with 30-day morbidity in 2010–2011 (n = 5,570). The resulting risk model’s discrimination and calibration were tested against the NSQIP-supplied morbidity model, and performance was validated against independent 2005–2009 data.

Results

Overall morbidity for proctectomy in 2010–2011 was 40.2 %; significantly higher than the 23.0 % rate predicted by the NSQIP-provided general and vascular surgery risk model. Frequent complications included bleeding (16.3 %), superficial infection (9.2 %), and sepsis (7.4 %). Our novel model incorporating 17 preoperative variables provided better discrimination and calibration (p < 0.05) than the NSQIP model and was validated against the 2005–2009 data. A web-based calculator makes this new model available for prospective risk assessment.

Conclusions

We conclude that the NSQIP-supplied risk model underestimates proctectomy morbidity and that this new, validated risk model and risk prediction tool (http://​myweb.​uiowa.​edu/​sksherman) may allow clinicians to counsel patients with accurate risk estimates using data available in the preoperative setting.
Literature
1.
go back to reference Markus PM, Martell J, Leister I, Horstmann O, Brinker J, Becker H. Predicting postoperative morbidity by clinical assessment. Br J Surg. 2005 Jan;92(1):101-6. Markus PM, Martell J, Leister I, Horstmann O, Brinker J, Becker H. Predicting postoperative morbidity by clinical assessment. Br J Surg. 2005 Jan;92(1):101-6.
2.
go back to reference Ferguson MK, Stromberg JD, Celauro AD. Estimating lung resection risk: a pilot study of trainee and practicing surgeons. Ann Thorac Surg. 2010 Apr;89(4):1037-42; discussion 42-3. Ferguson MK, Stromberg JD, Celauro AD. Estimating lung resection risk: a pilot study of trainee and practicing surgeons. Ann Thorac Surg. 2010 Apr;89(4):1037-42; discussion 42-3.
3.
go back to reference Volk ML, Roney M, Merion RM. Systematic bias in surgeons’ predictions of the donor-specific risk of liver transplant graft failure. Liver Transpl. 2013 Jun 19. Volk ML, Roney M, Merion RM. Systematic bias in surgeons’ predictions of the donor-specific risk of liver transplant graft failure. Liver Transpl. 2013 Jun 19.
4.
go back to reference Karliczek A, Harlaar NJ, Zeebregts CJ, Wiggers T, Baas PC, van Dam GM. Surgeons lack predictive accuracy for anastomotic leakage in gastrointestinal surgery. Int J Colorectal Dis. 2009 May;24(5):569-76. Karliczek A, Harlaar NJ, Zeebregts CJ, Wiggers T, Baas PC, van Dam GM. Surgeons lack predictive accuracy for anastomotic leakage in gastrointestinal surgery. Int J Colorectal Dis. 2009 May;24(5):569-76.
5.
go back to reference Merkow RP, Bilimoria KY, Ko CY. Surgical quality measurement: an evolving science. JAMA Surg. 2013 Jul 1;148(7):586-7. Merkow RP, Bilimoria KY, Ko CY. Surgical quality measurement: an evolving science. JAMA Surg. 2013 Jul 1;148(7):586-7.
6.
go back to reference Ragg JL, Watters DA, Guest GD. Preoperative risk stratification for mortality and major morbidity in major colorectal surgery. Dis Colon Rectum. 2009 Jul;52(7):1296-303. Ragg JL, Watters DA, Guest GD. Preoperative risk stratification for mortality and major morbidity in major colorectal surgery. Dis Colon Rectum. 2009 Jul;52(7):1296-303.
7.
go back to reference Fazio VW, Tekkis PP, Remzi F, Lavery IC. Assessment of operative risk in colorectal cancer surgery: the Cleveland Clinic Foundation colorectal cancer model. Dis Colon Rectum. 2004 Dec;47(12):2015-24. Fazio VW, Tekkis PP, Remzi F, Lavery IC. Assessment of operative risk in colorectal cancer surgery: the Cleveland Clinic Foundation colorectal cancer model. Dis Colon Rectum. 2004 Dec;47(12):2015-24.
8.
go back to reference Cohen ME, Bilimoria KY, Ko CY, Hall BL. Development of an American College of Surgeons National Surgery Quality Improvement Program: morbidity and mortality risk calculator for colorectal surgery. J Am Coll Surg. 2009 Jun;208(6):1009-16. Cohen ME, Bilimoria KY, Ko CY, Hall BL. Development of an American College of Surgeons National Surgery Quality Improvement Program: morbidity and mortality risk calculator for colorectal surgery. J Am Coll Surg. 2009 Jun;208(6):1009-16.
10.
go back to reference Raval MV, Cohen ME, Ingraham AM, Dimick JB, Osborne NH, Hamilton BH, Ko CY, Hall BL. Improving American College of Surgeons National Surgical Quality Improvement Program risk adjustment: incorporation of a novel procedure risk score. J Am Coll Surg. 2010 Dec;211(6):715-23. Raval MV, Cohen ME, Ingraham AM, Dimick JB, Osborne NH, Hamilton BH, Ko CY, Hall BL. Improving American College of Surgeons National Surgical Quality Improvement Program risk adjustment: incorporation of a novel procedure risk score. J Am Coll Surg. 2010 Dec;211(6):715-23.
11.
go back to reference Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko C, Cohen ME. Development and Evaluation of the Universal ACS NSQIP Surgical Risk Calculator: A Decision Aide and Informed Consent Tool for Patients and Surgeons. J Am Coll Surg. 2013;10.1016/j.jamcollsurg.2013.07.385. Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko C, Cohen ME. Development and Evaluation of the Universal ACS NSQIP Surgical Risk Calculator: A Decision Aide and Informed Consent Tool for Patients and Surgeons. J Am Coll Surg. 2013;10.​1016/​j.​jamcollsurg.​2013.​07.​385.
12.
go back to reference Balentine CJ, Robinson CN, Marshall CR, Wilks J, Buitrago W, Haderxhanaj K, Sansgiry S, Petersen NJ, Bansal V, Albo D, Berger DH. Waist circumference predicts increased complications in rectal cancer surgery. J Gastrointest Surg. 2010 Nov;14(11):1669-79. Balentine CJ, Robinson CN, Marshall CR, Wilks J, Buitrago W, Haderxhanaj K, Sansgiry S, Petersen NJ, Bansal V, Albo D, Berger DH. Waist circumference predicts increased complications in rectal cancer surgery. J Gastrointest Surg. 2010 Nov;14(11):1669-79.
13.
go back to reference Ballian N, Yamane B, Leverson G, Harms B, Heise CP, Foley EF, Kennedy GD. Body mass index does not affect postoperative morbidity and oncologic outcomes of total mesorectal excision for rectal adenocarcinoma. Ann Surg Oncol. 2010 Jun;17(6):1606-13. Ballian N, Yamane B, Leverson G, Harms B, Heise CP, Foley EF, Kennedy GD. Body mass index does not affect postoperative morbidity and oncologic outcomes of total mesorectal excision for rectal adenocarcinoma. Ann Surg Oncol. 2010 Jun;17(6):1606-13.
14.
go back to reference Canedo JA, Pinto RA, McLemore EC, Rosen L, Wexner SD. Restorative proctectomy with ileal pouch-anal anastomosis in obese patients. Dis Colon Rectum. 2010 Jul;53(7):1030-4. Canedo JA, Pinto RA, McLemore EC, Rosen L, Wexner SD. Restorative proctectomy with ileal pouch-anal anastomosis in obese patients. Dis Colon Rectum. 2010 Jul;53(7):1030-4.
15.
go back to reference Cohen ME, Ko CY, Bilimoria KY, Zhou L, Huffman K, Wang X, Liu Y, Kraemer K, Meng X, Merkow R, Chow W, Matel B, Richards K, Hart AJ, Dimick JB, Hall BL. Optimizing ACS NSQIP Modeling for Evaluation of Surgical Quality and Risk: Patient Risk Adjustment, Procedure Mix Adjustment, Shrinkage Adjustment, and Surgical Focus. J Am Coll Surg. 2013 Aug;217(2):336-46 e1. Cohen ME, Ko CY, Bilimoria KY, Zhou L, Huffman K, Wang X, Liu Y, Kraemer K, Meng X, Merkow R, Chow W, Matel B, Richards K, Hart AJ, Dimick JB, Hall BL. Optimizing ACS NSQIP Modeling for Evaluation of Surgical Quality and Risk: Patient Risk Adjustment, Procedure Mix Adjustment, Shrinkage Adjustment, and Surgical Focus. J Am Coll Surg. 2013 Aug;217(2):336-46 e1.
16.
go back to reference Hrabe JE, Sherman SK, Charlton ME, Cromwell JW, Byrn JC. The Effect of BMI on Outcomes in Proctectomy. Dis Colon Rectum. 2013; In Press. Bold indicates co-first authorship. Hrabe JE, Sherman SK, Charlton ME, Cromwell JW, Byrn JC. The Effect of BMI on Outcomes in Proctectomy. Dis Colon Rectum. 2013; In Press. Bold indicates co-first authorship.
17.
go back to reference User Guide for the 2011 Participant Use Data File. Chicago, IL: American College of Surgeons National Surgical Quality Improvement Program; October, 2012. User Guide for the 2011 Participant Use Data File. Chicago, IL: American College of Surgeons National Surgical Quality Improvement Program; October, 2012.
18.
go back to reference Fang SH, Cromwell JW, Wilkins KB, Eisenstat TE, Notaro JR, Alva S, Bustami R, Chinn BT. Is the abdominal repair of rectal prolapse safer than perineal repair in the highest risk patients? An NSQIP analysis. Dis Colon Rectum. 2012 Nov;55(11):1167-72. Fang SH, Cromwell JW, Wilkins KB, Eisenstat TE, Notaro JR, Alva S, Bustami R, Chinn BT. Is the abdominal repair of rectal prolapse safer than perineal repair in the highest risk patients? An NSQIP analysis. Dis Colon Rectum. 2012 Nov;55(11):1167-72.
19.
go back to reference Obesity: preventing and managing the global epidemic. Report of a WHO consultation. World Health Organ Tech Rep Ser. 2000;894:i-xii, 1-253. Obesity: preventing and managing the global epidemic. Report of a WHO consultation. World Health Organ Tech Rep Ser. 2000;894:i-xii, 1-253.
20.
go back to reference Hochberg Y, Benjamini Y. More powerful procedures for multiple significance testing. Stat Med. 1990 Jul;9(7):811-8. Hochberg Y, Benjamini Y. More powerful procedures for multiple significance testing. Stat Med. 1990 Jul;9(7):811-8.
21.
go back to reference Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, Muller M. pROC: an open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, Muller M. pROC: an open-source package for R and S + to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.
22.
go back to reference Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982 Apr;143(1):29-36. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982 Apr;143(1):29-36.
23.
go back to reference Brier GW. Verification of forecasts expressed in terms of probability. Monthly Weather Review. 1950 2/10/1950;78:1-3. Brier GW. Verification of forecasts expressed in terms of probability. Monthly Weather Review. 1950 2/10/1950;78:1-3.
24.
go back to reference Wilks DS. Statistical Methods in the Atmospheric Sciences.: Academic Press; 1995. Wilks DS. Statistical Methods in the Atmospheric Sciences.: Academic Press; 1995.
25.
go back to reference van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software. 2011 Dec;45(3):1-67. van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software. 2011 Dec;45(3):1-67.
26.
go back to reference Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited. Crit Care Med. 2007 Sep;35(9):2052-6. Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited. Crit Care Med. 2007 Sep;35(9):2052-6.
27.
go back to reference Paul P, Pennell ML, Lemeshow S. Standardizing the power of the Hosmer-Lemeshow goodness of fit test in large data sets. Stat Med. 2013 Jan 15;32(1):67-80. Paul P, Pennell ML, Lemeshow S. Standardizing the power of the Hosmer-Lemeshow goodness of fit test in large data sets. Stat Med. 2013 Jan 15;32(1):67-80.
28.
go back to reference Redelmeier DA, Bloch DA, Hickam DH. Assessing predictive accuracy: how to compare Brier scores. J Clin Epidemiol. 1991;44(11):1141-6. Redelmeier DA, Bloch DA, Hickam DH. Assessing predictive accuracy: how to compare Brier scores. J Clin Epidemiol. 1991;44(11):1141-6.
29.
go back to reference Ferro CAT. Comparing Probabilistic Forecasting Systems with the Brier Score. Weather and Forecasting. 2007 Oct. 2007;22(5):1076-88. Ferro CAT. Comparing Probabilistic Forecasting Systems with the Brier Score. Weather and Forecasting. 2007 Oct. 2007;22(5):1076-88.
30.
go back to reference Bradley AA, Schwartz SS, Hashino T. Sampling Uncertainty and Confidence Intervals for the Brier Score and Brier Skill Score. Weather and Forecasting. 2008 Oct. 2008;23(5):992-1006. Bradley AA, Schwartz SS, Hashino T. Sampling Uncertainty and Confidence Intervals for the Brier Score and Brier Skill Score. Weather and Forecasting. 2008 Oct. 2008;23(5):992-1006.
31.
go back to reference Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol. 2010 Aug;63(8):938-9; author reply 9. Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol. 2010 Aug;63(8):938-9; author reply 9.
32.
go back to reference Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007 Feb 20;115(7):928-35. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007 Feb 20;115(7):928-35.
33.
go back to reference Clark W, Siegel EM, Chen YA, Zhao X, Parsons CM, Hernandez JM, Weber J, Thareja S, Choi J, Shibata D. Quantitative measures of visceral adiposity and body mass index in predicting rectal cancer outcomes after neoadjuvant chemoradiation. J Am Coll Surg. 2013 Jun;216(6):1070-81. Clark W, Siegel EM, Chen YA, Zhao X, Parsons CM, Hernandez JM, Weber J, Thareja S, Choi J, Shibata D. Quantitative measures of visceral adiposity and body mass index in predicting rectal cancer outcomes after neoadjuvant chemoradiation. J Am Coll Surg. 2013 Jun;216(6):1070-81.
34.
go back to reference Amato A, Pescatori M. Perioperative blood transfusions for the recurrence of colorectal cancer. Cochrane Database Syst Rev. 2006(1):CD005033.PubMed Amato A, Pescatori M. Perioperative blood transfusions for the recurrence of colorectal cancer. Cochrane Database Syst Rev. 2006(1):CD005033.PubMed
35.
go back to reference Ball CG, Pitt HA, Kilbane ME, Dixon E, Sutherland FR, Lillemoe KD. Peri-operative blood transfusion and operative time are quality indicators for pancreatoduodenectomy. HPB (Oxford). 2010 Sep;12(7):465-71. Ball CG, Pitt HA, Kilbane ME, Dixon E, Sutherland FR, Lillemoe KD. Peri-operative blood transfusion and operative time are quality indicators for pancreatoduodenectomy. HPB (Oxford). 2010 Sep;12(7):465-71.
Metadata
Title
Development of an Improved Risk Calculator for Complications in Proctectomy
Authors
Scott K. Sherman
Jennifer E. Hrabe
Mary E. Charlton
John W. Cromwell
John C. Byrn
Publication date
01-05-2014
Publisher
Springer US
Published in
Journal of Gastrointestinal Surgery / Issue 5/2014
Print ISSN: 1091-255X
Electronic ISSN: 1873-4626
DOI
https://doi.org/10.1007/s11605-013-2448-2

Other articles of this Issue 5/2014

Journal of Gastrointestinal Surgery 5/2014 Go to the issue