Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2007

Open Access 01-12-2007 | Research article

Use of hierarchical models to evaluate performance of cardiac surgery centres in the Italian CABG outcome study

Authors: Paola D'Errigo, Maria E Tosti, Danilo Fusco, Carlo A Perucci, Fulvia Seccareccia, the research group of the Italian CABG Outcome Study

Published in: BMC Medical Research Methodology | Issue 1/2007

Login to get access

Abstract

Background

Hierarchical modelling represents a statistical method used to analyze nested data, as those concerning patients afferent to different hospitals. Aim of this paper is to build a hierarchical regression model using data from the "Italian CABG outcome study" in order to evaluate the amount of differences in adjusted mortality rates attributable to differences between centres.

Methods

The study population consists of all adult patients undergoing an isolated CABG between 2002–2004 in the 64 participating cardiac surgery centres.
A risk adjustment model was developed using a classical single-level regression. In the multilevel approach, the variable "clinical-centre" was employed as a group-level identifier. The intraclass correlation coefficient was used to estimate the proportion of variability in mortality between groups. Group-level residuals were adopted to evaluate the effect of clinical centre on mortality and to compare hospitals performance. Spearman correlation coefficient of ranks (ρ) was used to compare results from classical and hierarchical model.

Results

The study population was made of 34,310 subjects (mortality rate = 2.61%; range 0.33–7.63). The multilevel model estimated that 10.1% of total variability in mortality was explained by differences between centres. The analysis of group-level residuals highlighted 3 centres (VS 8 in the classical methodology) with estimated mortality rates lower than the mean and 11 centres (VS 7) with rates significantly higher. Results from the two methodologies were comparable (ρ = 0.99).

Conclusion

Despite known individual risk-factors were accounted for in the single-level model, the high variability explained by the variable "clinical-centre" states its importance in predicting 30-day mortality after CABG.
Appendix
Available only for authorised users
Literature
1.
go back to reference Iezzoni LI, Ash AS, Shwartz M, Daley J, Hughes JS, Mackiernan YD: Judging hospitals by severity-adjusted mortality rates : the influence of the severity-adjustment method. Am J Public Health. 1996, 86: 1379-87.CrossRefPubMedPubMedCentral Iezzoni LI, Ash AS, Shwartz M, Daley J, Hughes JS, Mackiernan YD: Judging hospitals by severity-adjusted mortality rates : the influence of the severity-adjustment method. Am J Public Health. 1996, 86: 1379-87.CrossRefPubMedPubMedCentral
2.
go back to reference Iezzoni LI: Risk Adjustment for Measuring Health Care Outcomes. 1997, Michigan: Health Administration Press Ann Arbor, 2 Iezzoni LI: Risk Adjustment for Measuring Health Care Outcomes. 1997, Michigan: Health Administration Press Ann Arbor, 2
3.
go back to reference Shroyer AL, Coombs LP, Peterson ED, Eiken MC, DeLong ER, Chen A, Ferguson TB, Grover FL, Edwards FH: Society of ThoracicSurgeons. The Society of Thoracic Surgeons: 30-day operativemortality and morbidity risk models. Ann Thorac Surg. 2003, 75 (6): 1856-64. 10.1016/S0003-4975(03)00179-6. discussion 1864–5CrossRefPubMed Shroyer AL, Coombs LP, Peterson ED, Eiken MC, DeLong ER, Chen A, Ferguson TB, Grover FL, Edwards FH: Society of ThoracicSurgeons. The Society of Thoracic Surgeons: 30-day operativemortality and morbidity risk models. Ann Thorac Surg. 2003, 75 (6): 1856-64. 10.1016/S0003-4975(03)00179-6. discussion 1864–5CrossRefPubMed
4.
go back to reference Agabiti N, Ancona C, Forestiere F, Arcà M, Perucci CA: Evaluating outcomes of hospital care following coronary artery bypass surgery in Roma, Italy. Eur J Cardiothorac Surg. 2003, 23: 599-760. 10.1016/S1010-7940(02)00866-7.CrossRefPubMed Agabiti N, Ancona C, Forestiere F, Arcà M, Perucci CA: Evaluating outcomes of hospital care following coronary artery bypass surgery in Roma, Italy. Eur J Cardiothorac Surg. 2003, 23: 599-760. 10.1016/S1010-7940(02)00866-7.CrossRefPubMed
5.
go back to reference New York State Department of Health. AdultCardiac Surgery in New York State 2001–2003. 2005, Albany, NY: New York State Department of Health New York State Department of Health. AdultCardiac Surgery in New York State 2001–2003. 2005, Albany, NY: New York State Department of Health
6.
go back to reference Health Grades. The Third Annual HealthGrades Hospital Quality and Clinical Excellence Study. 2005, Lakewood, CO Health Grades Health Grades. The Third Annual HealthGrades Hospital Quality and Clinical Excellence Study. 2005, Lakewood, CO Health Grades
7.
go back to reference Seccareccia F, Perucci CA, D'Errigo P, Arcà M, Fusco D, Rosato S, Greco D, on behalf of the research group of the Italian CABG Outcome Study: The 'Italian CABG Outcome Study': Short-term outcomes in patients with coronary artery bypass graft surgery. Eur J Cardiothoracic Surg. 2006, 29: 56-64. 10.1016/j.ejcts.2005.07.017.CrossRef Seccareccia F, Perucci CA, D'Errigo P, Arcà M, Fusco D, Rosato S, Greco D, on behalf of the research group of the Italian CABG Outcome Study: The 'Italian CABG Outcome Study': Short-term outcomes in patients with coronary artery bypass graft surgery. Eur J Cardiothoracic Surg. 2006, 29: 56-64. 10.1016/j.ejcts.2005.07.017.CrossRef
8.
go back to reference Greenland S: Principles of multilevel modelling. Int J Epidemiol. 2000, 29: 158-67. 10.1093/ije/29.1.158.CrossRefPubMed Greenland S: Principles of multilevel modelling. Int J Epidemiol. 2000, 29: 158-67. 10.1093/ije/29.1.158.CrossRefPubMed
9.
go back to reference Goldstein H, Thomas S: Using examination results as indicators of school and college performance. J R Statist Soc (Ser A). 1996, 159: 149-63. 10.2307/2983475.CrossRef Goldstein H, Thomas S: Using examination results as indicators of school and college performance. J R Statist Soc (Ser A). 1996, 159: 149-63. 10.2307/2983475.CrossRef
10.
go back to reference Diez-Roux AV: Multilevel analysis in public health research. Annu Rev Public Health. 2000, 21: 171-192. 10.1146/annurev.publhealth.21.1.171.CrossRefPubMed Diez-Roux AV: Multilevel analysis in public health research. Annu Rev Public Health. 2000, 21: 171-192. 10.1146/annurev.publhealth.21.1.171.CrossRefPubMed
11.
go back to reference Leyland A, Goldstein H: Multilevel modelling of health statistics. 2001, Wiley, New York Leyland A, Goldstein H: Multilevel modelling of health statistics. 2001, Wiley, New York
12.
go back to reference Bingenheimer JB, Raudenbush SW: Statistical and substantive inferences in Public Health: Issues in the Application of Multilevel Models. Annu Rev Public Health. 2004, 25: 53-77. 10.1146/annurev.publhealth.25.050503.153925.CrossRefPubMed Bingenheimer JB, Raudenbush SW: Statistical and substantive inferences in Public Health: Issues in the Application of Multilevel Models. Annu Rev Public Health. 2004, 25: 53-77. 10.1146/annurev.publhealth.25.050503.153925.CrossRefPubMed
13.
go back to reference Glance LG, Dick AW, Osler TM, Mukamel D: Using hierarchical modelling to measure ICU quality. Intensive Care Med. 2003, 29: 2223-2229. 10.1007/s00134-003-1959-9.CrossRefPubMed Glance LG, Dick AW, Osler TM, Mukamel D: Using hierarchical modelling to measure ICU quality. Intensive Care Med. 2003, 29: 2223-2229. 10.1007/s00134-003-1959-9.CrossRefPubMed
14.
go back to reference DeLong ER, Peterson ED, DeLong DM, Muhlbaier LH, Hackett S, Mark DB: Comparing risk-adjustment methods for provider profiling. Statist Med. 1997, 16: 2645-2664. 10.1002/(SICI)1097-0258(19971215)16:23<2645::AID-SIM696>3.0.CO;2-D.CrossRef DeLong ER, Peterson ED, DeLong DM, Muhlbaier LH, Hackett S, Mark DB: Comparing risk-adjustment methods for provider profiling. Statist Med. 1997, 16: 2645-2664. 10.1002/(SICI)1097-0258(19971215)16:23<2645::AID-SIM696>3.0.CO;2-D.CrossRef
15.
go back to reference Chrisiansen CL, Morris CN: Improving the statistical approach to health care provider profiling. Ann Intern Med. 1997, 127 (2): 764-768.CrossRef Chrisiansen CL, Morris CN: Improving the statistical approach to health care provider profiling. Ann Intern Med. 1997, 127 (2): 764-768.CrossRef
16.
go back to reference Luft S, Brown B: Calculating the probability of rare events: why settle for an sppoximation?. Health Serv Res. 1993, 28: 419-39.PubMedPubMedCentral Luft S, Brown B: Calculating the probability of rare events: why settle for an sppoximation?. Health Serv Res. 1993, 28: 419-39.PubMedPubMedCentral
17.
go back to reference McCulloch CE, Searle SR: Generalized, linear, and mixed models. 2001, John Wiley & Sons McCulloch CE, Searle SR: Generalized, linear, and mixed models. 2001, John Wiley & Sons
18.
go back to reference Rabe-Hesketh S, Skrondal A, Pickles A: Reliable Estimation of generalised linear mixed models using adaptive quadrature. The Stata Journal. 2002, 2: 1-21. Rabe-Hesketh S, Skrondal A, Pickles A: Reliable Estimation of generalised linear mixed models using adaptive quadrature. The Stata Journal. 2002, 2: 1-21.
19.
go back to reference Snijders T, Bosker R: Discrete dependent variables. Multilevel analysis: an introduction to basic and advanced multilevel modelling. 2000, London: Sage Publication Snijders T, Bosker R: Discrete dependent variables. Multilevel analysis: an introduction to basic and advanced multilevel modelling. 2000, London: Sage Publication
20.
go back to reference Goldstein H, Healy MJR: The graphical presentation of a collection of means. J R Statist Soc. 1995, 158: 175-177. 10.2307/2983411.CrossRef Goldstein H, Healy MJR: The graphical presentation of a collection of means. J R Statist Soc. 1995, 158: 175-177. 10.2307/2983411.CrossRef
22.
go back to reference Merlo J, Chaix B, Yang M, Lynch J, Rastman L: A brief conceptual tutorial of multilevel analysis in social epidemiology: linking the statistical concept of clustering to the idea of contextual phenomenon. J Epidemiol Community Health. 2005, 59: 443-449. 10.1136/jech.2004.023473.CrossRefPubMedPubMedCentral Merlo J, Chaix B, Yang M, Lynch J, Rastman L: A brief conceptual tutorial of multilevel analysis in social epidemiology: linking the statistical concept of clustering to the idea of contextual phenomenon. J Epidemiol Community Health. 2005, 59: 443-449. 10.1136/jech.2004.023473.CrossRefPubMedPubMedCentral
23.
go back to reference Hannan EL, Wu C, DeLong EL, Raudenbush SW: Predicting risk-adjusted mortality for CABG surgery. Logistic versus hierarchical models. Med Care. 2005, 43 (7): 726-735. 10.1097/01.mlr.0000167802.27044.44.CrossRefPubMed Hannan EL, Wu C, DeLong EL, Raudenbush SW: Predicting risk-adjusted mortality for CABG surgery. Logistic versus hierarchical models. Med Care. 2005, 43 (7): 726-735. 10.1097/01.mlr.0000167802.27044.44.CrossRefPubMed
24.
go back to reference Tekkis PP, McCulloch P, Steger AC, Irving SB, Jan DP: Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data. BMJ. 2003, 326: 786-788. 10.1136/bmj.326.7393.786.CrossRefPubMedPubMedCentral Tekkis PP, McCulloch P, Steger AC, Irving SB, Jan DP: Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data. BMJ. 2003, 326: 786-788. 10.1136/bmj.326.7393.786.CrossRefPubMedPubMedCentral
25.
go back to reference Coory M, Gibberd R: New measures for reporting the magnitude of small-area variation in rates. Statist Med. 1998, 17: 2625-2634. 10.1002/(SICI)1097-0258(19981130)17:22<2625::AID-SIM957>3.0.CO;2-4.CrossRef Coory M, Gibberd R: New measures for reporting the magnitude of small-area variation in rates. Statist Med. 1998, 17: 2625-2634. 10.1002/(SICI)1097-0258(19981130)17:22<2625::AID-SIM957>3.0.CO;2-4.CrossRef
26.
go back to reference Krogstad U, Hofoss D, Veenstra M, Gulbrandsen P, Hjortdahl P: Hospital quality improvement in context: a multilevel analysis of staff job evaluations. Qual Saf Health Care. 2005, 14: 438-442. 10.1136/qshc.2005.014233.CrossRefPubMedPubMedCentral Krogstad U, Hofoss D, Veenstra M, Gulbrandsen P, Hjortdahl P: Hospital quality improvement in context: a multilevel analysis of staff job evaluations. Qual Saf Health Care. 2005, 14: 438-442. 10.1136/qshc.2005.014233.CrossRefPubMedPubMedCentral
27.
go back to reference Grassetti L, Battauz M: Day hospital versus ordinary hospitalization: factors in treatment discrimination. Working paper series. 2004, Department of Statistical Sciences, University of Padua, Italy, N. 7, July 2004 Grassetti L, Battauz M: Day hospital versus ordinary hospitalization: factors in treatment discrimination. Working paper series. 2004, Department of Statistical Sciences, University of Padua, Italy, N. 7, July 2004
28.
go back to reference Shahian DM, Normand SL, Torchiana DF, Lewis SM, Pastore JO, Kuntz RE, Dreyer PI: Cardiac surgery report cards: review and statistical critic. Ann Thorac Surg. 2001, 72: 2155-2168. 10.1016/S0003-4975(01)03222-2.CrossRefPubMed Shahian DM, Normand SL, Torchiana DF, Lewis SM, Pastore JO, Kuntz RE, Dreyer PI: Cardiac surgery report cards: review and statistical critic. Ann Thorac Surg. 2001, 72: 2155-2168. 10.1016/S0003-4975(01)03222-2.CrossRefPubMed
29.
go back to reference New Jersey Department of Health and SeniorServices. Cardiac surgery in New Jersey 2001. 2004, Trenton, NJ:New Jersey Department of Health and Senior Services New Jersey Department of Health and SeniorServices. Cardiac surgery in New Jersey 2001. 2004, Trenton, NJ:New Jersey Department of Health and Senior Services
30.
go back to reference Office of Statewide Health Planning &Development. The California report on coronary artery bypassgraft surgery, 1999 hospital data. San Francisco. 2003, CA: Office of Statewide Health Planning & Development Office of Statewide Health Planning &Development. The California report on coronary artery bypassgraft surgery, 1999 hospital data. San Francisco. 2003, CA: Office of Statewide Health Planning & Development
31.
go back to reference Pennsylvania Health Care Cost Containment Council. Pennsylvania's guide to coronary artery bypass graft surgery 2002. 2004, Harrisburg, PA: Pennsylvania Health Care Cost Containment Council Pennsylvania Health Care Cost Containment Council. Pennsylvania's guide to coronary artery bypass graft surgery 2002. 2004, Harrisburg, PA: Pennsylvania Health Care Cost Containment Council
32.
go back to reference Austin PC, Tu JV, Alter DA: Comparing Hierarchical Modelling with traditional logistic regression analysis among patients hospitalized with acute myocardial infarction: should we be analyzing cardiovascular outcomes data differently?. Am Heart J. 145 (1): 26-35. Austin PC, Tu JV, Alter DA: Comparing Hierarchical Modelling with traditional logistic regression analysis among patients hospitalized with acute myocardial infarction: should we be analyzing cardiovascular outcomes data differently?. Am Heart J. 145 (1): 26-35.
Metadata
Title
Use of hierarchical models to evaluate performance of cardiac surgery centres in the Italian CABG outcome study
Authors
Paola D'Errigo
Maria E Tosti
Danilo Fusco
Carlo A Perucci
Fulvia Seccareccia
the research group of the Italian CABG Outcome Study
Publication date
01-12-2007
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2007
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-7-29

Other articles of this Issue 1/2007

BMC Medical Research Methodology 1/2007 Go to the issue