Skip to main content
Top
Published in: BMC Health Services Research 1/2023

Open Access 01-12-2023 | Abdominal Aortic Aneurysm | Research

Improving hospital quality risk-adjustment models using interactions identified by hierarchical group lasso regularisation

Authors: Monika Ray, Sharon Zhao, Sheng Wang, Alex Bohl, Patrick S. Romano

Published in: BMC Health Services Research | Issue 1/2023

Login to get access

Abstract

Background

Risk-adjustment (RA) models are used to account for severity of illness in comparing patient outcomes across hospitals. Researchers specify covariates as main effects, but they often ignore interactions or use stratification to account for effect modification, despite limitations due to rare events and sparse data. Three Agency for Healthcare Research and Quality (AHRQ) hospital-level Quality Indicators currently use stratified models, but their variable performance and limited interpretability motivated the design of better models.

Methods

We analysed patient discharge de-identified data from 14 State Inpatient Databases, AHRQ Healthcare Cost and Utilization Project, California Department of Health Care Access and Information, and New York State Department of Health. We used hierarchical group lasso regularisation (HGLR) to identify first-order interactions in several AHRQ inpatient quality indicators (IQI) - IQI 09 (Pancreatic Resection Mortality Rate), IQI 11 (Abdominal Aortic Aneurysm Repair Mortality Rate), and Patient Safety Indicator 14 (Postoperative Wound Dehiscence Rate). These models were compared with stratum-specific and composite main effects models with covariates selected by least absolute shrinkage and selection operator (LASSO).

Results

HGLR identified clinically meaningful interactions for all models. Synergistic IQI 11 interactions, such as between hypertension and respiratory failure, suggest patients who merit special attention in perioperative care. Antagonistic IQI 11 interactions, such as between shock and chronic comorbidities, illustrate that naïve main effects models overestimate risk in key subpopulations. Interactions for PSI 14 suggest key subpopulations for whom the risk of wound dehiscence is similar between open and laparoscopic approaches, whereas laparoscopic approach is safer for other groups. Model performance was similar or superior for composite models with HGLR-selected features, compared to those with LASSO-selected features.

Conclusions

In this application to high-profile, high-stakes risk-adjustment models, HGLR selected interactions that maintained or improved model performance in populations with heterogeneous risk, while identifying clinically important interactions. The HGLR package is scalable to handle a large number of covariates and their interactions and is customisable to use multiple CPU cores to reduce analysis time. The HGLR method will allow scholars to avoid creating stratified models on sparse data, improve model calibration, and reduce bias. Future work involves testing using other combinations of risk factors, such as vital signs and laboratory values. Our study focuses on a real-world problem of considerable importance to hospitals and policy-makers who must use RA models for statutorily mandated public reporting and payment programmes.
Literature
9.
go back to reference Barrera-Gomez J, Agier L, Portengen L, Chadeau-Hyam M, Giorgis-Allemand L, Siroux V, et al. A systematic comparison of statistical methods to detect interactions in exposome-health associations. Environ Health. 2017;16:1–13.CrossRef Barrera-Gomez J, Agier L, Portengen L, Chadeau-Hyam M, Giorgis-Allemand L, Siroux V, et al. A systematic comparison of statistical methods to detect interactions in exposome-health associations. Environ Health. 2017;16:1–13.CrossRef
10.
go back to reference Boltz MM, Hollenbeak CS, Ortenzi G, Dillon PW. Synergistic implications of multiple postoperative outcomes. Am J Med Qual. 2012;27(5):383–90.CrossRefPubMed Boltz MM, Hollenbeak CS, Ortenzi G, Dillon PW. Synergistic implications of multiple postoperative outcomes. Am J Med Qual. 2012;27(5):383–90.CrossRefPubMed
15.
go back to reference Cox D. Two further applications of a model for a method of binary regression. Biometrika. 1958;45:562–5. Cox D. Two further applications of a model for a method of binary regression. Biometrika. 1958;45:562–5.
16.
go back to reference Davis J, Goadrich M. The relationship between precision-recall and roc curves. Proceedings of the 23rd International Conference on Machine Learning (ICML2006). Association for Computing Machinery; 2006. Davis J, Goadrich M. The relationship between precision-recall and roc curves. Proceedings of the 23rd International Conference on Machine Learning (ICML2006). Association for Computing Machinery; 2006.
17.
go back to reference Finazzi S, Poole D, Luciani D, Cogo P, Bertolini G. Calibration Belt for Quality-of-Care Assessment Based on Dichotomous Outcomes. PLoS ONE. 2011;6(2):e16110.CrossRefPubMedPubMedCentral Finazzi S, Poole D, Luciani D, Cogo P, Bertolini G. Calibration Belt for Quality-of-Care Assessment Based on Dichotomous Outcomes. PLoS ONE. 2011;6(2):e16110.CrossRefPubMedPubMedCentral
18.
go back to reference Firth D. Bias reduction of maximum likelihood estimates. Biometrika. 1993;80:27–38.CrossRef Firth D. Bias reduction of maximum likelihood estimates. Biometrika. 1993;80:27–38.CrossRef
21.
go back to reference Hosmer DW, Hjort NL. Goodness-of-fit processes for logistic regression: simulation results. Stat Med. 2002;21(18):2723–38.CrossRefPubMed Hosmer DW, Hjort NL. Goodness-of-fit processes for logistic regression: simulation results. Stat Med. 2002;21(18):2723–38.CrossRefPubMed
22.
go back to reference Hsu D. Identifying key variables and interactions in statistical models of building energy consumption using regularization. Energy. 2015;83:144–55.CrossRef Hsu D. Identifying key variables and interactions in statistical models of building energy consumption using regularization. Energy. 2015;83:144–55.CrossRef
23.
go back to reference Kim M, Brady JE, Li G. Interaction effects of acute kidney injury, acute respiratory failure, and sepsis on 30-day postoperative mortality in patients undergoing high-risk intraabdominal general surgical procedures. Anesth Analg. 2015;121(6):1536–46.CrossRefPubMed Kim M, Brady JE, Li G. Interaction effects of acute kidney injury, acute respiratory failure, and sepsis on 30-day postoperative mortality in patients undergoing high-risk intraabdominal general surgical procedures. Anesth Analg. 2015;121(6):1536–46.CrossRefPubMed
24.
go back to reference Kim M, Li G. Two-way interaction effects of perioperative complications on 30-day mortality in general surgery. World J Surg. 2018;42(1):2–11.CrossRefPubMed Kim M, Li G. Two-way interaction effects of perioperative complications on 30-day mortality in general surgery. World J Surg. 2018;42(1):2–11.CrossRefPubMed
25.
go back to reference Lezzoni L. Risk Adjustment for Measuring Health Care Outcomes, Chapter 12. 4th ed. Health Administration Press; 2012. Lezzoni L. Risk Adjustment for Measuring Health Care Outcomes, Chapter 12. 4th ed. Health Administration Press; 2012.
27.
go back to reference Locham S, Rizwan M, Dakour-Aridi H, Faateh M, Nejim B, Malas M. Outcomes after elective abdominal aortic aneurysm repair in obese versus nonobese patients. J Vasc Surg. 2018;68(6):1696–705.CrossRefPubMed Locham S, Rizwan M, Dakour-Aridi H, Faateh M, Nejim B, Malas M. Outcomes after elective abdominal aortic aneurysm repair in obese versus nonobese patients. J Vasc Surg. 2018;68(6):1696–705.CrossRefPubMed
28.
go back to reference Low Y, Daugherty A, Schroeder E, Chen W, Seto T, Weber S, et al. Synergistic drug combinations from electronic health records and gene expression. J Am Med Inform Assoc. 2016;24:565–76.CrossRefPubMedCentral Low Y, Daugherty A, Schroeder E, Chen W, Seto T, Weber S, et al. Synergistic drug combinations from electronic health records and gene expression. J Am Med Inform Assoc. 2016;24:565–76.CrossRefPubMedCentral
29.
go back to reference Lunardon N, Menardi G, Torelli N. ROSE: a Package for Binary Imbalanced Learning. R J. 2014;6(1):79–89.CrossRef Lunardon N, Menardi G, Torelli N. ROSE: a Package for Binary Imbalanced Learning. R J. 2014;6(1):79–89.CrossRef
30.
go back to reference Menardi G, Torelli N. Training and assessing classification rules with imbalanced data. Data Min Knowl Disc. 2014;28:92–122. Menardi G, Torelli N. Training and assessing classification rules with imbalanced data. Data Min Knowl Disc. 2014;28:92–122.
31.
go back to reference Merath K, Chen Q, Bagante F, Akgul O, Idrees JJ, Dillhoff M, et al. Synergistic Effects of Perioperative Complications on 30-Day Mortality Following Hepatopancreatic Surgery. J Gastrointest Surg. 2018;22(10):1715–23.CrossRefPubMed Merath K, Chen Q, Bagante F, Akgul O, Idrees JJ, Dillhoff M, et al. Synergistic Effects of Perioperative Complications on 30-Day Mortality Following Hepatopancreatic Surgery. J Gastrointest Surg. 2018;22(10):1715–23.CrossRefPubMed
32.
go back to reference Nattino G, Finazzi S, Bertolini G. A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes. Stat Med. 2014;33(14):2390–407.CrossRefPubMed Nattino G, Finazzi S, Bertolini G. A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes. Stat Med. 2014;33(14):2390–407.CrossRefPubMed
33.
go back to reference Nattino G, Lemeshow S, Phillips G, Finazzi S, Bertolini G. Assessing the Calibration of Dichotomous Outcome Models with the Calibration Belt. Stata J. 2017;17(4):1003–14.CrossRef Nattino G, Lemeshow S, Phillips G, Finazzi S, Bertolini G. Assessing the Calibration of Dichotomous Outcome Models with the Calibration Belt. Stata J. 2017;17(4):1003–14.CrossRef
34.
go back to reference Paul P, Pennell M, Lemeshow S. Standardizing the power of the Hosmer–lemeshow goodness of fit test in large data sets. Stat Med. 2012;32(1):67–80.CrossRefPubMed Paul P, Pennell M, Lemeshow S. Standardizing the power of the Hosmer–lemeshow goodness of fit test in large data sets. Stat Med. 2012;32(1):67–80.CrossRefPubMed
35.
go back to reference Pindyck R, Rubinfeld D. Econometric Models and Economic Forecasts, Chapter 5. 2nd ed. McGraw-Hill; 1981. Pindyck R, Rubinfeld D. Econometric Models and Economic Forecasts, Chapter 5. 2nd ed. McGraw-Hill; 1981.
37.
go back to reference Puhr R, Heinze G, Nold M, Lusa L, Geroldinger A. Firth’s logistic regression with rare events: accurate effect estimates and predictions? Stat Med. 2017;36:2302–17.CrossRefPubMed Puhr R, Heinze G, Nold M, Lusa L, Geroldinger A. Firth’s logistic regression with rare events: accurate effect estimates and predictions? Stat Med. 2017;36:2302–17.CrossRefPubMed
38.
go back to reference Rose S, McGuire TG. Limitations of P-Values and R-squared for Stepwise Regression Building: a Fairness Demonstration in Health Policy Risk Adjustment. Am Stat. 2019;73(Suppl 1):152–6.CrossRefPubMedPubMedCentral Rose S, McGuire TG. Limitations of P-Values and R-squared for Stepwise Regression Building: a Fairness Demonstration in Health Policy Risk Adjustment. Am Stat. 2019;73(Suppl 1):152–6.CrossRefPubMedPubMedCentral
39.
go back to reference Saito T, Rehmsmeier M. The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE. 2015;10:e0118432.CrossRefPubMedPubMedCentral Saito T, Rehmsmeier M. The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE. 2015;10:e0118432.CrossRefPubMedPubMedCentral
41.
go back to reference Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol. 1996;58(1):267–88. Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Methodol. 1996;58(1):267–88.
42.
go back to reference Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B. 2006;68:49–67.CrossRef Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B. 2006;68:49–67.CrossRef
Metadata
Title
Improving hospital quality risk-adjustment models using interactions identified by hierarchical group lasso regularisation
Authors
Monika Ray
Sharon Zhao
Sheng Wang
Alex Bohl
Patrick S. Romano
Publication date
01-12-2023
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2023
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/s12913-023-10423-9

Other articles of this Issue 1/2023

BMC Health Services Research 1/2023 Go to the issue