Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2008

Open Access 01-12-2008 | Research article

Bayes rules for optimally using Bayesian hierarchical regression models in provider profiling to identify high-mortality hospitals

Author: Peter C Austin

Published in: BMC Medical Research Methodology | Issue 1/2008

Login to get access

Abstract

Background

There is a growing trend towards the production of "hospital report-cards" in which hospitals with higher than acceptable mortality rates are identified. Several commentators have advocated for the use of Bayesian hierarchical models in provider profiling. Several researchers have shown that some degree of misclassification will result when hospital report cards are produced. The impact of misclassifying hospital performance can be quantified using different loss functions.

Methods

We propose several families of loss functions for hospital report cards and then develop Bayes rules for these families of loss functions. The resultant Bayes rules minimize the expected loss arising from misclassifying hospital performance. We develop Bayes rules for generalized 1-0 loss functions, generalized absolute error loss functions, and for generalized squared error loss functions. We then illustrate the application of these decision rules on a sample of 19,757 patients hospitalized with an acute myocardial infarction at 163 hospitals.

Results

We found that the number of hospitals classified as having higher than acceptable mortality is affected by the relative penalty assigned to false negatives compared to false positives. However, the choice of loss function family had a lesser impact upon which hospitals were identified as having higher than acceptable mortality.

Conclusion

The design of hospital report cards can be placed in a decision-theoretic framework. This allows researchers to minimize costs arising from the misclassification of hospitals. The choice of loss function can affect the classification of a small number of hospitals.
Appendix
Available only for authorised users
Literature
1.
go back to reference Luft HS, Romano PS, Remy LL, Rainwater J: Annual report of the California Hospital Outcomes Project. 1993, Sacramento, CA: California Office of Statewide Health Planning and Development Luft HS, Romano PS, Remy LL, Rainwater J: Annual report of the California Hospital Outcomes Project. 1993, Sacramento, CA: California Office of Statewide Health Planning and Development
2.
go back to reference Pennsylvania Health Care Cost Containment Council: Focus on heart attack in Pennsylvania. Research methods and results. 1996, Harrisburg, PA: Pennsylvania Health Care Cost Containment Council Pennsylvania Health Care Cost Containment Council: Focus on heart attack in Pennsylvania. Research methods and results. 1996, Harrisburg, PA: Pennsylvania Health Care Cost Containment Council
3.
go back to reference Scottish Office: Clinical outcome indicators, 1994. 1995, Edinburgh: Clinical Resource and Audit Group Scottish Office: Clinical outcome indicators, 1994. 1995, Edinburgh: Clinical Resource and Audit Group
4.
go back to reference Tu JV, Austin P, Naylor CD, Iron K, Zhang H: Acute myocardial infarction outcomes in Ontario. Cardiovascular health services in Ontario: an ICES atlas. Edited by: Naylor CD, Slaugher PM. 1999, Toronto, Ontario: Institute for Clinical Evaluative Sciences, 83-110. Tu JV, Austin P, Naylor CD, Iron K, Zhang H: Acute myocardial infarction outcomes in Ontario. Cardiovascular health services in Ontario: an ICES atlas. Edited by: Naylor CD, Slaugher PM. 1999, Toronto, Ontario: Institute for Clinical Evaluative Sciences, 83-110.
5.
go back to reference Jacobs FM: Cardiac Surgery in New Jersey in 2002: A Consumer Report. 2005, Trenton, NJ: New Jersey Department of Health and Senior Services Jacobs FM: Cardiac Surgery in New Jersey in 2002: A Consumer Report. 2005, Trenton, NJ: New Jersey Department of Health and Senior Services
6.
go back to reference New York State Department of Health: Coronary artery bypass graft surgery in New York State 1989–1991. 1992, Albany, NY: New York State Department of Health New York State Department of Health: Coronary artery bypass graft surgery in New York State 1989–1991. 1992, Albany, NY: New York State Department of Health
7.
go back to reference Pennsylvania Health Care Cost Containment Council: A Consumer Guide to Coronary Artery Bypass Graft Surgery. 1995, Harrisburg, PA: Pennsylvania Health Care Cost Containment Council, 4: Pennsylvania Health Care Cost Containment Council: A Consumer Guide to Coronary Artery Bypass Graft Surgery. 1995, Harrisburg, PA: Pennsylvania Health Care Cost Containment Council, 4:
8.
go back to reference Massachusetts Data Analysis Center: Adult Coronary Artery Bypass Graft Surgery in the Commonwealth of Massachusetts, Fiscal Year 2006 Report (October 1, 2005 – September 30, 2006). Hospital and Surgeon Standardized 30-day Mortality rates. 2008, Boston, MA: Department of Health Care Policy, Harvard Medical School Massachusetts Data Analysis Center: Adult Coronary Artery Bypass Graft Surgery in the Commonwealth of Massachusetts, Fiscal Year 2006 Report (October 1, 2005 – September 30, 2006). Hospital and Surgeon Standardized 30-day Mortality rates. 2008, Boston, MA: Department of Health Care Policy, Harvard Medical School
9.
go back to reference Naylor CD, Rothwell DM, Tu JV, Austin PC, the Cardiac Care Network Steering Committee: Outcomes of Coronary Artery Bypass Graft Surgery in Ontario. Cardiovascular health services in Ontario: an ICES atlas. Edited by: Naylor CD, Slaughter PM. 1999, Toronto, Ontario: Institute for Clinical Evaluative Sciences: Toronto, Canada, 189-197. Naylor CD, Rothwell DM, Tu JV, Austin PC, the Cardiac Care Network Steering Committee: Outcomes of Coronary Artery Bypass Graft Surgery in Ontario. Cardiovascular health services in Ontario: an ICES atlas. Edited by: Naylor CD, Slaughter PM. 1999, Toronto, Ontario: Institute for Clinical Evaluative Sciences: Toronto, Canada, 189-197.
10.
go back to reference McGlynn EA: Introduction and overview of the conceptual framework for a national quality measurement and reporting system. Medical Care. 2003, 41: 1-I. 10.1097/00005650-200301000-00001.CrossRef McGlynn EA: Introduction and overview of the conceptual framework for a national quality measurement and reporting system. Medical Care. 2003, 41: 1-I. 10.1097/00005650-200301000-00001.CrossRef
11.
go back to reference Normand SLT, Shahian DM: Statistical and clinical aspects of hospital outcomes profiling. Statistical Science. 2007, 22: 206-226. 10.1214/088342307000000096.CrossRef Normand SLT, Shahian DM: Statistical and clinical aspects of hospital outcomes profiling. Statistical Science. 2007, 22: 206-226. 10.1214/088342307000000096.CrossRef
12.
go back to reference Austin PC, Alter DA, Tu JV: The accuracy of fixed and random effects models in calculating risk-adjusted mortality rates: A Monte Carlo assessment. Medical Decision Making. 2003, 23: 526-539. 10.1177/0272989X03258443.CrossRefPubMed Austin PC, Alter DA, Tu JV: The accuracy of fixed and random effects models in calculating risk-adjusted mortality rates: A Monte Carlo assessment. Medical Decision Making. 2003, 23: 526-539. 10.1177/0272989X03258443.CrossRefPubMed
13.
go back to reference Austin PC, Brunner LJ: Optimal Bayesian probability levels for hospital report Cards. Health Services and Outcomes Research Methodology. 10.1007/s10742-007-0025-4. Austin PC, Brunner LJ: Optimal Bayesian probability levels for hospital report Cards. Health Services and Outcomes Research Methodology. 10.1007/s10742-007-0025-4.
14.
go back to reference Thomas JW, Hofer TP: Accuracy of risk-adjusted mortality rates as a measure of hospital quality of care. Medical Care. 1999, 37: 83-92. 10.1097/00005650-199901000-00012.CrossRefPubMed Thomas JW, Hofer TP: Accuracy of risk-adjusted mortality rates as a measure of hospital quality of care. Medical Care. 1999, 37: 83-92. 10.1097/00005650-199901000-00012.CrossRefPubMed
15.
go back to reference Burgess JF, Christiansen CL, Michalak SE, Morris CN: Medical profiling: improving standards and risk adjustments using hierarchical models. Journal of Health Economics. 2000, 19: 291-309. 10.1016/S0167-6296(99)00034-X.CrossRefPubMed Burgess JF, Christiansen CL, Michalak SE, Morris CN: Medical profiling: improving standards and risk adjustments using hierarchical models. Journal of Health Economics. 2000, 19: 291-309. 10.1016/S0167-6296(99)00034-X.CrossRefPubMed
16.
go back to reference Normand SL, Glickman ME, Gatsonis CA: Statistical methods for profiling providers of medical care: issues and applications. Journal of the American Statistical Association. 1997, 92: 803-814. 10.2307/2965545.CrossRef Normand SL, Glickman ME, Gatsonis CA: Statistical methods for profiling providers of medical care: issues and applications. Journal of the American Statistical Association. 1997, 92: 803-814. 10.2307/2965545.CrossRef
17.
go back to reference Christiansen CL, Morris CN: Improving the statistical approach to health care provider profiling. Annals of Internal Medicine. 1997, 127: 764-768.CrossRefPubMed Christiansen CL, Morris CN: Improving the statistical approach to health care provider profiling. Annals of Internal Medicine. 1997, 127: 764-768.CrossRefPubMed
18.
go back to reference Spiegelhalter DJ, Aylin P, Best NG, Evans SJW, Murray GD: Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry. J R Statist Soc A. 2000, 165: 191-231. 10.1111/1467-985X.02021.CrossRef Spiegelhalter DJ, Aylin P, Best NG, Evans SJW, Murray GD: Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry. J R Statist Soc A. 2000, 165: 191-231. 10.1111/1467-985X.02021.CrossRef
19.
go back to reference Thomas N, Longford NT, Rolph JE: Empirical Bayes methods for estimating hospital-specific mortality rates. Statistics in Medicine. 1994, 13: 889-903. 10.1002/sim.4780130902.CrossRefPubMed Thomas N, Longford NT, Rolph JE: Empirical Bayes methods for estimating hospital-specific mortality rates. Statistics in Medicine. 1994, 13: 889-903. 10.1002/sim.4780130902.CrossRefPubMed
20.
go back to reference Austin PC: The reliability and validity of Bayesian methods for hospital profiling: A Monte Carlo assessment. Journal of Statistical Planning and Inference. 2005, 128: 109-122. 10.1016/j.jspi.2003.10.006.CrossRef Austin PC: The reliability and validity of Bayesian methods for hospital profiling: A Monte Carlo assessment. Journal of Statistical Planning and Inference. 2005, 128: 109-122. 10.1016/j.jspi.2003.10.006.CrossRef
21.
go back to reference Austin PC: A comparison of Bayesian methods for profiling hospital performance. Medical Decision Making. 2002, 22: 163-172. 10.1177/02729890222063044.CrossRefPubMed Austin PC: A comparison of Bayesian methods for profiling hospital performance. Medical Decision Making. 2002, 22: 163-172. 10.1177/02729890222063044.CrossRefPubMed
22.
go back to reference Gajewski BJ, Petroski G, Thompson S, Dunton N, Wrona M, Becker A, Coffland V: Letter to the editor: the effect of provider-level ascertainment bias on profiling nursing homes by Roy J, Mor V. Statistics in Medicine. 2006, 25: 1976-1977. 10.1002/sim.2480.CrossRefPubMed Gajewski BJ, Petroski G, Thompson S, Dunton N, Wrona M, Becker A, Coffland V: Letter to the editor: the effect of provider-level ascertainment bias on profiling nursing homes by Roy J, Mor V. Statistics in Medicine. 2006, 25: 1976-1977. 10.1002/sim.2480.CrossRefPubMed
23.
go back to reference DeGroot MH: Optimal Statistical Decisions. 1970, New York, NY: McGraw-Hill DeGroot MH: Optimal Statistical Decisions. 1970, New York, NY: McGraw-Hill
24.
go back to reference Berger JO: Statistical Decision Theory and Bayesian Analysis. 1980, New York, NY: Springer-VerlagCrossRef Berger JO: Statistical Decision Theory and Bayesian Analysis. 1980, New York, NY: Springer-VerlagCrossRef
25.
go back to reference Tu JV, Naylor CD, Austin P: Temporal changes in the outcomes of acute myocardial infarction in Ontario, 1992–96. Canadian Medical Association Journal. 1999, 161: 1257-1261.PubMedPubMedCentral Tu JV, Naylor CD, Austin P: Temporal changes in the outcomes of acute myocardial infarction in Ontario, 1992–96. Canadian Medical Association Journal. 1999, 161: 1257-1261.PubMedPubMedCentral
26.
go back to reference Tu JV, Austin PC, Walld R, Roos L, Agras J, McDonald KM: Development and validation of the Ontario acute myocardial mortality prediction rules. Journal of the American College of Cardiology. 2001, 37: 992-7. 10.1016/S0735-1097(01)01109-3.CrossRefPubMed Tu JV, Austin PC, Walld R, Roos L, Agras J, McDonald KM: Development and validation of the Ontario acute myocardial mortality prediction rules. Journal of the American College of Cardiology. 2001, 37: 992-7. 10.1016/S0735-1097(01)01109-3.CrossRefPubMed
27.
go back to reference Tu JV, Austin PC, Chan BTB: Relationship between annual volume of patients treated by admitting physician and mortality after acute myocardial infarction. Journal of the American Medical Association. 2001, 285: 3116-3122. 10.1001/jama.285.24.3116.CrossRefPubMed Tu JV, Austin PC, Chan BTB: Relationship between annual volume of patients treated by admitting physician and mortality after acute myocardial infarction. Journal of the American Medical Association. 2001, 285: 3116-3122. 10.1001/jama.285.24.3116.CrossRefPubMed
28.
go back to reference Alter DA, Naylor CD, Austin PC, Tu JV: Long-term MI outcomes at hospitals with or without on-site revascularization. Journal of the American Medical Association. 2001, 285: 2101-8. 10.1001/jama.285.16.2101.CrossRefPubMed Alter DA, Naylor CD, Austin PC, Tu JV: Long-term MI outcomes at hospitals with or without on-site revascularization. Journal of the American Medical Association. 2001, 285: 2101-8. 10.1001/jama.285.16.2101.CrossRefPubMed
29.
go back to reference Gilks WR, Richardson S, Spiegelhalter DJ: Introducing Markov chain Monte Carlo. Markov chain Monte Carlo in practice. Edited by: Gilks WR, Richardson S, Spiegelhalter DJ. 1996, London, UK: Chapman & Hall, 1-19. Gilks WR, Richardson S, Spiegelhalter DJ: Introducing Markov chain Monte Carlo. Markov chain Monte Carlo in practice. Edited by: Gilks WR, Richardson S, Spiegelhalter DJ. 1996, London, UK: Chapman & Hall, 1-19.
30.
go back to reference Gilks WR, Thomas A, Spiegelhalter DJ: A language and program for complex Bayesian modelling. The Statistician. 1994, 43: 169-78. 10.2307/2348941.CrossRef Gilks WR, Thomas A, Spiegelhalter DJ: A language and program for complex Bayesian modelling. The Statistician. 1994, 43: 169-78. 10.2307/2348941.CrossRef
31.
go back to reference Gelman A, Rubin DB: Inference from iterative simulation using multiple sequences. Statistical Sciences. 1992, 7: 457-472. 10.1214/ss/1177011136.CrossRef Gelman A, Rubin DB: Inference from iterative simulation using multiple sequences. Statistical Sciences. 1992, 7: 457-472. 10.1214/ss/1177011136.CrossRef
32.
go back to reference Iezzoni LI, Ed: Risk adjustment for measuring healthcare outcomes. 1997, Chicago, IL: Health Administration Press, Second Iezzoni LI, Ed: Risk adjustment for measuring healthcare outcomes. 1997, Chicago, IL: Health Administration Press, Second
33.
go back to reference Austin PC, Anderson GM: Optimal statistical decisions for hospital report cards. Medical Decision Making. 2005, 25: 11-19. 10.1177/0272989X04273142.CrossRefPubMed Austin PC, Anderson GM: Optimal statistical decisions for hospital report cards. Medical Decision Making. 2005, 25: 11-19. 10.1177/0272989X04273142.CrossRefPubMed
Metadata
Title
Bayes rules for optimally using Bayesian hierarchical regression models in provider profiling to identify high-mortality hospitals
Author
Peter C Austin
Publication date
01-12-2008
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2008
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-8-30

Other articles of this Issue 1/2008

BMC Medical Research Methodology 1/2008 Go to the issue