Skip to main content
Top
Published in: BMC Health Services Research 1/2010

Open Access 01-12-2010 | Research article

Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture

Authors: Joann S Sorra, Naomi Dyer

Published in: BMC Health Services Research | Issue 1/2010

Login to get access

Abstract

Background

The Agency for Healthcare Research and Quality (AHRQ) Hospital Survey on Patient Safety Culture was designed to assess staff views on patient safety culture in hospital settings. The purpose of this study was to examine the multilevel psychometric properties of the survey.

Methods

Survey data from 331 U.S. hospitals with 2,267 hospital units and 50,513 respondents were analyzed to examine the psychometric properties of the survey's items and composites. Item factor loadings, intraclass correlations (ICCs), design effects, internal consistency reliabilities, and multilevel confirmatory factor analyses (MCFA) were examined as well as intercorrelations among the survey's composites.

Results

Psychometric analyses confirmed the multilevel nature of the data at the individual, unit and hospital levels of analysis. Results provided overall evidence supporting the 12 dimensions and 42 items included in the AHRQ Hospital Survey on Patient Safety Culture as having acceptable psychometric properties at all levels of analysis, with a few exceptions. The Staffing composite fell slightly below cutoffs in a number of areas, but is conceptually important given its impact on patient safety. In addition, one hospital-level model fit indicator for the Supervisor/Manager Expectations & Actions Promoting Patient Safety composite was low (CFI = .82), but all other psychometrics for this scale were good. Average dimension intercorrelations were moderate at .42 at the individual level, .50 at the unit level, and .56 at the hospital level.

Conclusions

Psychometric analyses conducted on a very large database of hospitals provided overall support for the patient safety culture dimensions and items included in the AHRQ Hospital Survey on Patient Safety Culture. The survey's items and dimensions overall are psychometrically sound at the individual, unit, and hospital levels of analysis and can be used by researchers and hospitals interested in assessing patient safety culture. Further research is needed to study the criterion-related validity of the survey by analysing the relationship between patient safety culture and patient outcomes and studying how to improve patient safety culture.
Appendix
Available only for authorised users
Literature
1.
go back to reference Lee T, Harrison K: Assessing safety culture in nuclear power stations. Safety Science. 2002, 34: 61-97. 10.1016/S0925-7535(00)00007-2.CrossRef Lee T, Harrison K: Assessing safety culture in nuclear power stations. Safety Science. 2002, 34: 61-97. 10.1016/S0925-7535(00)00007-2.CrossRef
2.
go back to reference Perrow C: Normal Accidents: Living with High Risk Technologies. 1984, New York: Basic Books Perrow C: Normal Accidents: Living with High Risk Technologies. 1984, New York: Basic Books
3.
go back to reference Reason J: Managing the Risks of Organizational Accidents. 1997, Aldershot, UK: Ashgate Reason J: Managing the Risks of Organizational Accidents. 1997, Aldershot, UK: Ashgate
4.
go back to reference Weick KE: Organizational Culture as a Source of High Reliability. California Management Review. 1987, 24 (2): 112-127.CrossRef Weick KE: Organizational Culture as a Source of High Reliability. California Management Review. 1987, 24 (2): 112-127.CrossRef
5.
go back to reference Zohar D: Safety climate in industrial organizations: Theoretical and applied implications. Journal of Applied Psychology. 1980, 65: 96-102. 10.1037/0021-9010.65.1.96.CrossRefPubMed Zohar D: Safety climate in industrial organizations: Theoretical and applied implications. Journal of Applied Psychology. 1980, 65: 96-102. 10.1037/0021-9010.65.1.96.CrossRefPubMed
6.
go back to reference Frankel AS, Leonard MW, Denham CR, Fair and Just Culture, Team Behavior and Leadership Engagement: The Tools to Achieve High Reliability. Health Services Research. 2006, 41 (4, Part II): 1690-1617. 10.1111/j.1475-6773.2006.00572.x.CrossRefPubMedPubMedCentral Frankel AS, Leonard MW, Denham CR, Fair and Just Culture, Team Behavior and Leadership Engagement: The Tools to Achieve High Reliability. Health Services Research. 2006, 41 (4, Part II): 1690-1617. 10.1111/j.1475-6773.2006.00572.x.CrossRefPubMedPubMedCentral
7.
go back to reference Pronovost PJ, Berenholtz SM, Goeschel CA, Needham DM, Sexton JB, Thompson DA, Lubomski LH, Marsteller JA, Makary MA, Hunt E: Creating High Reliability in Health Care Organizations. Health Services Research. 2006, 41 (4, Part II): 1599-1617. 10.1111/j.1475-6773.2006.00567.x.CrossRefPubMedPubMedCentral Pronovost PJ, Berenholtz SM, Goeschel CA, Needham DM, Sexton JB, Thompson DA, Lubomski LH, Marsteller JA, Makary MA, Hunt E: Creating High Reliability in Health Care Organizations. Health Services Research. 2006, 41 (4, Part II): 1599-1617. 10.1111/j.1475-6773.2006.00567.x.CrossRefPubMedPubMedCentral
9.
go back to reference Institute of Medicine: To Err is Human: Building a Safer Health System. Edited by: Kohn LT, Corrigan JM, Donaldson MS. 1999, Washington, DC: National Academies Press Institute of Medicine: To Err is Human: Building a Safer Health System. Edited by: Kohn LT, Corrigan JM, Donaldson MS. 1999, Washington, DC: National Academies Press
11.
go back to reference Jeffe DB, Dunagan WC, Garbutt J, Burroughs TE, Gallagher TH, Hill PR, Harris CB, Bommarito K, Fraser V: Using focus groups to understand physicians' and nurses' perspectives on error reporting in hospitals. Joint Commission Journal on Quality and Safety. 2004, 30 (9): 471-479.PubMed Jeffe DB, Dunagan WC, Garbutt J, Burroughs TE, Gallagher TH, Hill PR, Harris CB, Bommarito K, Fraser V: Using focus groups to understand physicians' and nurses' perspectives on error reporting in hospitals. Joint Commission Journal on Quality and Safety. 2004, 30 (9): 471-479.PubMed
12.
go back to reference Farley DO, Haviland A, Champagne S, Jain AK, Battles JB, Munier WB, Loeb JM: Adverse-event-reporting practices by US hospitals: results of a national survey. Quality and Safety in Health Care. 2008, 17: 416-423. 10.1136/qshc.2007.024638.CrossRefPubMed Farley DO, Haviland A, Champagne S, Jain AK, Battles JB, Munier WB, Loeb JM: Adverse-event-reporting practices by US hospitals: results of a national survey. Quality and Safety in Health Care. 2008, 17: 416-423. 10.1136/qshc.2007.024638.CrossRefPubMed
13.
go back to reference Harper ML, Helmreich RL: Identifying Barriers to the Success of a Reporting System. Advances in Patient Safety: From Research to Implementation. AHRQ Publication Nos. 050021 (1-4). Agency for Healthcare Research and Quality, Rockville, MD. 2005, 3. Harper ML, Helmreich RL: Identifying Barriers to the Success of a Reporting System. Advances in Patient Safety: From Research to Implementation. AHRQ Publication Nos. 050021 (1-4). Agency for Healthcare Research and Quality, Rockville, MD. 2005, 3.
14.
go back to reference Nieva VF, Sorra J: Safety culture assessment: a tool for improving patient safety in healthcare organizations. Quality and Safety in Healthcare. 2003, 12 (Suppl 2): 17-23. Nieva VF, Sorra J: Safety culture assessment: a tool for improving patient safety in healthcare organizations. Quality and Safety in Healthcare. 2003, 12 (Suppl 2): 17-23.
15.
go back to reference Scott T, Mannion R, Davies H, Marshall M: The Quantitative Measurement of Organizational Culture in Health Care: A Review of the Available Instruments. Health Serv Res. 2003, 38 (3): 923-945. 10.1111/1475-6773.00154.CrossRefPubMedPubMedCentral Scott T, Mannion R, Davies H, Marshall M: The Quantitative Measurement of Organizational Culture in Health Care: A Review of the Available Instruments. Health Serv Res. 2003, 38 (3): 923-945. 10.1111/1475-6773.00154.CrossRefPubMedPubMedCentral
16.
go back to reference Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts P, Thomas E: The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Services Research. 2006, 6: 44-10.1186/1472-6963-6-44.CrossRefPubMedPubMedCentral Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts P, Thomas E: The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Services Research. 2006, 6: 44-10.1186/1472-6963-6-44.CrossRefPubMedPubMedCentral
17.
go back to reference Singer SJ, Gaba DM, Geppert JJ, Sinaiko AD, Howard SK, Park KC: The culture of safety: results of an organization-wide survey in 15 California hospitals. Quality and Safety in Health Care. 2003, 12: 112-118. 10.1136/qhc.12.2.112.CrossRefPubMedPubMedCentral Singer SJ, Gaba DM, Geppert JJ, Sinaiko AD, Howard SK, Park KC: The culture of safety: results of an organization-wide survey in 15 California hospitals. Quality and Safety in Health Care. 2003, 12: 112-118. 10.1136/qhc.12.2.112.CrossRefPubMedPubMedCentral
18.
go back to reference Colla JB, Bracken AC, Kinney LM, Weeks WB: Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005, 14: 364-366. 10.1136/qshc.2005.014217.CrossRefPubMedPubMedCentral Colla JB, Bracken AC, Kinney LM, Weeks WB: Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005, 14: 364-366. 10.1136/qshc.2005.014217.CrossRefPubMedPubMedCentral
19.
go back to reference Flin RH, Burns C, Mearns K, Yule S, Robertson EM: Measuring hospital safety climate. Quality and Safety in Health Care. 2006, 15: 109-115. 10.1136/qshc.2005.014761.CrossRefPubMedPubMedCentral Flin RH, Burns C, Mearns K, Yule S, Robertson EM: Measuring hospital safety climate. Quality and Safety in Health Care. 2006, 15: 109-115. 10.1136/qshc.2005.014761.CrossRefPubMedPubMedCentral
20.
go back to reference Singla AK, Kitch BT, Weissman JS, Campbell EG: Assessing patient safety culture: a review and synthesis of the measurement tools. J of Patient Saf. 2006, 2 (3): 105-115. 10.1097/01.jps.0000235388.39149.5a.CrossRef Singla AK, Kitch BT, Weissman JS, Campbell EG: Assessing patient safety culture: a review and synthesis of the measurement tools. J of Patient Saf. 2006, 2 (3): 105-115. 10.1097/01.jps.0000235388.39149.5a.CrossRef
21.
go back to reference Sorra JS, Nieva VF: Hospital Survey on Patient Safety Culture. (Prepared by Westat, under Contract No. 290-96-0004). 2004, Rockville, MD: Agency for Healthcare Research and Quality Sorra JS, Nieva VF: Hospital Survey on Patient Safety Culture. (Prepared by Westat, under Contract No. 290-96-0004). 2004, Rockville, MD: Agency for Healthcare Research and Quality
22.
go back to reference Sorra JS, Nieva VF: Pilot study: reliability and validity of the hospital survey on patient safety. Technical report prepared by Westat under Contract No. 290-96-004. 2004, Rockville, MD: Agency for Healthcare Research and Quality Sorra JS, Nieva VF: Pilot study: reliability and validity of the hospital survey on patient safety. Technical report prepared by Westat under Contract No. 290-96-004. 2004, Rockville, MD: Agency for Healthcare Research and Quality
23.
go back to reference Sorra J, Nieva V, Famolaro T, Dyer N: Hospital survey on patient safety culture: 2007 comparative database report. (Prepared by Westat, Rockville, MD, under contract No. 233-02-0087, Task Order No. 18). 2007, Rockville, MD: Agency for Healthcare Research and Quality Sorra J, Nieva V, Famolaro T, Dyer N: Hospital survey on patient safety culture: 2007 comparative database report. (Prepared by Westat, Rockville, MD, under contract No. 233-02-0087, Task Order No. 18). 2007, Rockville, MD: Agency for Healthcare Research and Quality
24.
go back to reference Peterson RA: Marketing Letters. 2000, 11 (3): 261-275. 10.1023/A:1008191211004.CrossRef Peterson RA: Marketing Letters. 2000, 11 (3): 261-275. 10.1023/A:1008191211004.CrossRef
25.
go back to reference Muthen BO, Muthen LK: Mplus: User's Guide. 1998, Los Angeles: Muthen & Muthen Muthen BO, Muthen LK: Mplus: User's Guide. 1998, Los Angeles: Muthen & Muthen
26.
go back to reference Hu LT, Bentler PM: Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999, 6: 1-55. 10.1080/10705519909540118.CrossRef Hu LT, Bentler PM: Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999, 6: 1-55. 10.1080/10705519909540118.CrossRef
28.
go back to reference Nunnally JC, Bernstein IH: Psychometric Theory. 1994, New York: McGraw Hill Nunnally JC, Bernstein IH: Psychometric Theory. 1994, New York: McGraw Hill
29.
go back to reference O'Brien , Robert M: A Caution Regarding Rules of Thumb for Variance Inflation Factors. Quality and Quantity. 2007, 41 (5): 673-690. 10.1007/s11135-006-9018-6.CrossRef O'Brien , Robert M: A Caution Regarding Rules of Thumb for Variance Inflation Factors. Quality and Quantity. 2007, 41 (5): 673-690. 10.1007/s11135-006-9018-6.CrossRef
30.
go back to reference Institute of Medicine: Keeping Patients Safe: Transforming the Work Environment of Nurses. 2003, Washington, DC: National Academies Press Institute of Medicine: Keeping Patients Safe: Transforming the Work Environment of Nurses. 2003, Washington, DC: National Academies Press
31.
go back to reference Westrum R: A Typology of Organisational Cultures. Quality and Safety in Health Care. 2004, 13 (Suppl II): ii22-ii27. 10.1136/qshc.2003.009522.PubMedPubMedCentral Westrum R: A Typology of Organisational Cultures. Quality and Safety in Health Care. 2004, 13 (Suppl II): ii22-ii27. 10.1136/qshc.2003.009522.PubMedPubMedCentral
32.
go back to reference Patient Safety Indicators Overview: AHRQ Quality Indicators. 2004, Agency for Healthcare Research and Quality, Rockville, MD Patient Safety Indicators Overview: AHRQ Quality Indicators. 2004, Agency for Healthcare Research and Quality, Rockville, MD
33.
go back to reference Resar RK, Rozich JD: Measurement of harm with trigger tools. Quality and Safety in Healthcare. 2003, 12 (Supplement 2): ii39-45. Resar RK, Rozich JD: Measurement of harm with trigger tools. Quality and Safety in Healthcare. 2003, 12 (Supplement 2): ii39-45.
34.
go back to reference Marx D, Slonim AD: Assessing Patient Safety Risk Before the Injury Occurs: An Introduction to Sociotechnical Probabilistic Risk Modelling in Health Care. Quality and Safety in Healthcare. 2003, 12 (Supplement 2): ii33-38. Marx D, Slonim AD: Assessing Patient Safety Risk Before the Injury Occurs: An Introduction to Sociotechnical Probabilistic Risk Modelling in Health Care. Quality and Safety in Healthcare. 2003, 12 (Supplement 2): ii33-38.
35.
go back to reference Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK: AHRQ's hospital survey on patient safety culture: psychometric analyses. J Patient Saf. 2009, 5 (3): 139-44. 10.1097/PTS.0b013e3181b53f6e.CrossRefPubMed Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK: AHRQ's hospital survey on patient safety culture: psychometric analyses. J Patient Saf. 2009, 5 (3): 139-44. 10.1097/PTS.0b013e3181b53f6e.CrossRefPubMed
36.
go back to reference Smits M, Christiaans-Dingelhoff I, Wagner C, van der Wal G, Groenewegen PP: The psychometric properties of the 'Hospital Survey on Patient Safety Culture' in Dutch hospitals. BMC Health Services Research. 2008, 230-10.1186/1472-6963-8-230. 8 Smits M, Christiaans-Dingelhoff I, Wagner C, van der Wal G, Groenewegen PP: The psychometric properties of the 'Hospital Survey on Patient Safety Culture' in Dutch hospitals. BMC Health Services Research. 2008, 230-10.1186/1472-6963-8-230. 8
37.
go back to reference Smits M, Wagner C, Spreeuwenberg P, van der Wal G, Groenewegen PP: Measuring patient safety culture: an assessment of the clustering of responses at unit level and hospital level. Quality and Safety in Health Care. 2009, 292-296. 10.1136/qshc.2007.025965. 18 Smits M, Wagner C, Spreeuwenberg P, van der Wal G, Groenewegen PP: Measuring patient safety culture: an assessment of the clustering of responses at unit level and hospital level. Quality and Safety in Health Care. 2009, 292-296. 10.1136/qshc.2007.025965. 18
38.
go back to reference Olsen E: Reliability and validity of the Hospital Survey on Patient Safety Culture at a Norwegian hospital. Quality and Safety Improvement Research: Methods and Research Practice from the International Quality Improvement Research Network (QIRN). Edited by: Øvretveit J, Sousa PJ. 2008, Lisbon: National School of Public Health, 173-186. Olsen E: Reliability and validity of the Hospital Survey on Patient Safety Culture at a Norwegian hospital. Quality and Safety Improvement Research: Methods and Research Practice from the International Quality Improvement Research Network (QIRN). Edited by: Øvretveit J, Sousa PJ. 2008, Lisbon: National School of Public Health, 173-186.
Metadata
Title
Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture
Authors
Joann S Sorra
Naomi Dyer
Publication date
01-12-2010
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2010
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/1472-6963-10-199

Other articles of this Issue 1/2010

BMC Health Services Research 1/2010 Go to the issue