Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Health Services Research
Published in: BMC Health Services Research 1/2013

Open Access 01-12-2013 | Research article

Overall scores as an alternative to global ratings in patient experience surveys; a comparison of four methods

Authors: Maarten W Krol, Dolf de Boer, Jany JDJM Rademakers, Diana M Delnoij

Published in: BMC Health Services Research | Issue 1/2013

Login to get access

Abstract

Background

Global ratings of healthcare by patients are a popular way of summarizing patients’ experiences. Summary scores can be used for comparing healthcare provider performance and provider rankings. As an alternative, overall scores from actual patient experiences can be constructed as summary scores. This paper addresses the statistical and practical characteristics of overall scores as an alternative to a global rating in summarizing patient survey results.

Methods

Data from a 2010 patient experience survey for approximately 12,000 nursing home residents (7.5% of all Dutch nursing home residents at the time) from 464 nursing homes in the Netherlands (25% of the Dutch nursing homes) was used. Data was collected through specifically designed standardized interview surveys. The respondents’ scores for 15 established quality indicators (or composites) for nursing home care were used to calculate overall scores for each nursing home, using four different strategies. The characteristics of the overall scores were compared against each other and with the respondents’ global rating.

Results

The individual indicators showed stronger associations with each of the four overall strategies than with the global ratings. Furthermore, the dispersion of the overall scores across nursing homes was greater. Differences between overall scores appeared limited.

Conclusions

Overall scores proved more valid than global ratings as a summary of the indicator scores, and also showed more pronounced differences between nursing homes. Because of the limited statistical differences between the strategies, and for practical reasons, a straightforward averaging of quality indicator scores may be preferred as an overall score.
Appendix
Available only for authorised users
Literature
1.
Williamson C: The patient movement as an emancipation movement. Health Expect. 2008, 11: 102-112. 10.1111/j.1369-7625.2007.00475.x.CrossRefPubMed
2.
Delnoij DM: Measuring patient experiences in Europe: what can we learn from the experiences in the USA and England?. Eur J Public Health. 2009, 19: 354-356. 10.1093/eurpub/ckp105.CrossRefPubMed
3.
Maarse H, ter Meulen R: Consumer choice in Dutch health insurance after reform. Health Care Anal. 2006, 14: 37-49.CrossRefPubMed
4.
Zuidgeest M, Delnoij D, Luijkx KG, De Boer D, Westert GP: Patients’ experiences of the quality of long-term care among the elderly: comparing scores over time. BMC Health Serv Res. 2012, 12: 26-10.1186/1472-6963-12-26.CrossRefPubMedPubMedCentral
5.
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG: Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med. 2008, 148: 111-123. 10.7326/0003-4819-148-2-200801150-00006.CrossRefPubMed
6.
Damman OC, Hendriks M, Rademakers J, Spreeuwenberg P, Delnoij DM, Groenewegen PP: Consumers’ interpretation and use of comparative information on the quality of health care: the effect of presentation approaches. Health Expect. 2012, 15: 197-211. 10.1111/j.1369-7625.2011.00671.x.CrossRefPubMed
7.
Delnoij DM, Rademakers JJ, Groenewegen PP: The Dutch consumer quality index: an example of stakeholder involvement in indicator development. BMC Health Serv Res. 2010, 10: 88-10.1186/1472-6963-10-88.CrossRefPubMedPubMedCentral
8.
Zaslavsky AM, Beaulieu ND, Landon BE, Cleary PD: Dimensions of consumer-assessed quality of Medicare managed-care health plans. Med Care. 2000, 38: 162-174. 10.1097/00005650-200002000-00006.CrossRefPubMed
9.
Chang JT, Hays RD, Shekelle PG, MacLean CH, Solomon DH, Reuben DB, Roth CP, Kamberg CJ, Adams J, Young RT, Wenger NS: Patients’ Global ratings of their health care Are Not associated with the technical quality of their care. Ann Intern Med. 2006, 144: 665-672. 10.7326/0003-4819-144-9-200605020-00010.CrossRefPubMed
10.
Hibbard JH, Slovic P, Peters E, Finucane ML: Strategies for reporting health plan performance information to consumers: evidence from controlled studies. Health Serv Res. 2002, 37: 291-313. 10.1111/1475-6773.024.CrossRefPubMedPubMedCentral
11.
Hibbard JH, Peters E: Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Annu Rev Public Health. 2003, 24: 413-433. 10.1146/annurev.publhealth.24.100901.141005.CrossRefPubMed
12.
Ranganathan M, Hibbard J, Rodday AM, de Brantes F, Conroy K, Rogers WH, et al: Motivating public use of physician-level performance data: an experiment on the effects of message and mode. Med Care Res Rev. 2009, 66: 68-81.CrossRefPubMed
13.
Damman OC, Hendriks M, Rademakers J, Delnoij DM, Groenewegen PP: How do healthcare consumers process and evaluate comparative healthcare information? A qualitative study using cognitive interviews. BMC Public Health. 2009, 9: 423-10.1186/1471-2458-9-423.CrossRefPubMedPubMedCentral
14.
de Boer D, Delnoij D, Rademakers J: Do patient experiences on priority aspects of health care predict their global rating of quality of care? A study in five patient groups. Health Expect. 2010, 13: 285-297.PubMed
15.
Rademakers J, Delnoij D, De Boer D: Structure, process or outcome: which contributes most to patients’ overall assessment of healthcare quality?. BMJ Qual Saf. 2011, 20: 326-331. 10.1136/bmjqs.2010.042358.CrossRefPubMed
16.
Jacobs R, Goddard M, Smith PC: How robust are hospital ranks based on composite performance measures?. Med Care. 2005, 43: 1177-1184. 10.1097/01.mlr.0000185692.72905.4a.CrossRefPubMed
17.
18.
Triemstra AHM, Winters S, Kool RB, Wiegers TA: Measuring client experiences in long-term care in the netherlands: a pilot study with the consumer quality index long-term care. BMC Health Serv Res. 2010, 10: 95-10.1186/1472-6963-10-95.CrossRefPubMedPubMedCentral
19.
Delnoij DM, ten Asbroek G, Arah OA, de Koning JS, Stam P, Poll A, et al: Made in the USA: the import of American consumer assessment of health plan surveys (CAHPS) into the dutch social insurance system. Eur J Public Health. 2006, 16: 652-659. 10.1093/eurpub/ckl023.CrossRefPubMed
20.
van der Velden LFJ, Francke AL, Batenburg RS: Trends of supply and demand in nursing and care in the Netherlands. A synthesis of existing literature and data sources [in Dutch]. 2011, NIVEL: Utrecht
21.
Singer JD: Using SAS Proc Mixed to fit multilevel models, hierarchical models, and individual growth models. J Educat Behaviour Stat. 1998, 24: 325-355.
22.
Reeves MJ, Gargano J, Maier KS, Broderick JP, Frankel M, LaBresh KA, et al: Patient-level and hospital-level determinants of the quality of acute stroke care: a multilevel modeling approach. Stroke. 2010, 41: 2924-2931. 10.1161/STROKEAHA.110.598664.CrossRefPubMed
23.
Damman OC, Stubbe JH, Hendriks M, Arah OA, Spreeuwenberg P, Delnoij DM, et al: Using multilevel modeling to assess case-mix adjusters in consumer experience surveys in health care. Med Care. 2009, 47: 496-503. 10.1097/MLR.0b013e31818afa05.CrossRefPubMed
24.
Snijders TAB, Bosker RJ: Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. 1999, Thousand Oaks, CA: SAGE Publications, 1
25.
Zaslavsky AM, Zaborski LB, Ding L, Shaul JA, Cioffi MJ, Cleary PD: Adjusting performance measures to ensure equitable plan comparisons. Health Care Financ Rev. 2001, 22: 109-126.PubMedPubMedCentral
26.
Efron B, Morris C: Stein’s Paradox in statistics. Sci Am. 1977, 5: 119-127.CrossRef
27.
Casella G: An introduction to empirical bayes data analysis. Am Stat. 1985, 39: 83-87. 10.2307/2682801.
28.
Greenland S: Principles of multilevel modelling. Int J Epidemiol. 2000, 29: 158-167. 10.1093/ije/29.1.158.CrossRefPubMed
29.
Diez Roux AV: A glossary for multilevel analysis. J Epidemiol Community Health. 2002, 56: 588-594. 10.1136/jech.56.8.588.CrossRefPubMed
30.
Hays WL: Statistics. 1994, London: Holt, Rinehart Winston, 5
31.
Carey RG, Seibert JH: A patient survey system to measure quality improvement: questionnaire reliability and validity. Med Care. 1993, 31: 834-845. 10.1097/00005650-199309000-00008.CrossRefPubMed
32.
Stubbe JH, Brouwer W, Delnoij DM: Patients’ Experiences with quality of hospital care: the consumer quality index cataract questionnaire. BMC Ophthalmol. 2007, 7: 14-10.1186/1471-2415-7-14.CrossRefPubMedPubMedCentral
33.
Stubbe JH, Gelsema T, Delnoij DM: The consumer quality index Hip knee questionnaire measuring patients’ experiences with quality of care after a total hip or knee arthroplasty. BMC Health Serv Res. 2007, 7: 60-10.1186/1472-6963-7-60.CrossRefPubMedPubMedCentral
34.
Damman OC, Hendriks M, Sixma HJ: Towards more patient centred healthcare: a new consumer quality index instrument to assess patients’ experiences with breast care. Eur J Cancer. 2009, 45: 1569-1577. 10.1016/j.ejca.2008.12.011.CrossRefPubMed
35.
De Boer D, Delnoij D, Rademakers J: The discriminative power of patient experience surveys. BMC Health Serv Res. 2011, 11: 332-10.1186/1472-6963-11-332.CrossRefPubMedPubMedCentral
36.
Kirk SA, Campbell SM, Kennell-Webb S, Reeves D, Roland MO, Marshall MN: Assessing the quality of care of multiple conditions in general practice: practical and methodological problems. Qual Saf Health Care. 2003, 12: 421-427. 10.1136/qhc.12.6.421.CrossRefPubMedPubMedCentral
37.
Scholle SH, Roski J, Adams JL, Dunn DL, Kerr EA, Dugan DP, et al: Benchmarking physician performance: reliability of individual and composite measures. Am J Manag Care. 2008, 14: 833-838.PubMedPubMedCentral
38.
Kaplan SH, Griffith JL, Price LL, Pawlson LG, Greenfield S: Improving the reliability of physician performance assessment: identifying the ”physician effect“ on quality and creating composite measures. Med Care. 2009, 47: 378-387. 10.1097/MLR.0b013e31818dce07.CrossRefPubMed
39.
Reeves D, Campbell SM, Adams J, Shekelle PG, Kontopantelis E, Roland MO: Combining multiple indicators of clinical quality: an evaluation of different analytic approaches. Med Care. 2007, 45: 489-496. 10.1097/MLR.0b013e31803bb479.CrossRefPubMed
40.
Shwartz M, Ren J, Pekoz EA, Wang X, Cohen AB, Restuccia JD: Estimating a composite measure of hospital quality from the Hospital Compare database: differences when using a Bayesian hierarchical latent variable model versus denominator-based weights. Med Care. 2008, 46: 778-785. 10.1097/MLR.0b013e31817893dc.CrossRefPubMed
41.
Peterson ED, DeLong ER, Masoudi FA, O’Brien SM, Peterson PN, Rumsfeld JS, et al: ACCF/AHA 2010 Position Statement on Composite Measures for Healthcare Performance Assessment: a report of American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures (Writing Committee to Develop a Position Statement on Composite Measures). J Am Coll Cardiol. 2010, 55: 1755-1766. 10.1016/j.jacc.2010.02.016.CrossRefPubMed
42.
Holmboe ES, Weng W, Arnold GK, Kaplan SH, Normand SL, Greenfield S, et al: The comprehensive care project: measuring physician performance in ambulatory practice. Health Serv Res. 2010, 45: 1912-1933. 10.1111/j.1475-6773.2010.01160.x.CrossRefPubMedPubMedCentral
Metadata
Title
Overall scores as an alternative to global ratings in patient experience surveys; a comparison of four methods
Authors
Maarten W Krol
Dolf de Boer
Jany JDJM Rademakers
Diana M Delnoij
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2013
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/1472-6963-13-479

Other articles of this Issue 1/2013

BMC Health Services Research 1/2013 Go to the issue

Research article

Patients’ socioeconomic status and their evaluations of primary care in Hong Kong

Research article

Effects of health insurance on non-working married women’s medical care use and bed days at home

Study protocol

Transforming primary healthcare by including the stakeholders involved in delivering care to people living in poverty: EQUIhealThY study protocol

Research article

Wait times for publicly funded addiction and problem gambling treatment agencies in Ontario, Canada

Research article

Comparison of prothrombin time (INR) results and main characteristics of patients on warfarin treatment in primary health care centers and anticoagulation clinics

Study protocol

Multifaceted intervention to enhance the screening and care of hospitalised malnourished children: study protocol for the PREDIRE cluster randomized controlled trial