Skip to main content
Top
Published in: BMC Health Services Research 1/2016

Open Access 01-12-2016 | Research article

Evidence of non-response bias in the Press-Ganey patient satisfaction survey

Authors: A. R. Tyser, A. M. Abtahi, M. McFadden, A. P. Presson

Published in: BMC Health Services Research | Issue 1/2016

Login to get access

Abstract

Background

Measures of patient satisfaction have gained prominence in recent years as changes to the American health care system have led to the incorporation of such metrics into reimbursement models. The response rate for widely-used outpatient satisfaction metrics and variables influencing the probability of survey nonresponse remain largely unknown.

Methods

We reviewed all unique adult patients (16,779) who completed an outpatient encounter in the Department of Orthopaedic surgery at our academic institution from 1/1/13 to 10/24/13. Survey data was linked to each clinic visit, and patient factors including age, sex, insurance type, zip code, and orthopaedic subspecialty visited were recorded.
The overall survey response rate was calculated. Logistic regression was performed, and unadjusted and adjusted odds ratios of patients’ probability of responding to the Press-Ganey survey were calculated.

Results

Two thousand seven hundred sixty two (16.5 %) of individuals completed a Press-Ganey patient satisfaction survey and 14017 patients did not respond. For those patients considered responders, 906 patients (32.8 %) did not complete all the survey items. Among these 906 patients, the mean number of missing items was 2.24 (Standard Deviation SD: 2.19).
Age, sex, insurance type, and orthopaedic subspecialty were all found to be associated with the odds of responding to our patient satisfaction survey. Advancing age increased the odds of responding to the survey (Adjusted Odds Ratio (OR) = 3.396 for ≥65 vs. 18–29, p < 0.001). Several variables were associated with a decreased odds of survey response, and included male sex (Adjusted OR = 0.782 for Males vs. Females, p < 0.001), insurance type (Adjusted OR = 0.311 for Medicaid/Self-Pay vs. Private), and subspecialty type (Adjusted OR = 0.623 for Trauma vs. Adult Reconstruction).

Conclusions

The response rate to the Press-Ganey Medical Practice Survey of outpatient satisfaction is low in an orthopaedic outpatient population, and furthermore, is impacted by patient characteristics such as age, sex, insurance type, and type of orthopaedic subspecialist encountered. The findings of the present study should inform future non-response weighting procedures in this area. More research is needed to assess non-response bias—including follow-up studies of non-respondents—in order to more accurately measure of patient satisfaction.
Literature
1.
go back to reference Fenton JJ, Jerant AF, Bertakis KD, Franks P. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch Intern Med. 2012;172:405–11.CrossRefPubMed Fenton JJ, Jerant AF, Bertakis KD, Franks P. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch Intern Med. 2012;172:405–11.CrossRefPubMed
2.
go back to reference Glickman SW, Boulding W, Manary M, et al. Patient satisfaction and its relationship with clinical quality and inpatient mortality in acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2010;3:188–95.CrossRefPubMed Glickman SW, Boulding W, Manary M, et al. Patient satisfaction and its relationship with clinical quality and inpatient mortality in acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2010;3:188–95.CrossRefPubMed
3.
go back to reference Manary MP, Boulding W, Staelin R, Glickman SW. The patient experience and health outcomes. N Engl J Med. 2013;368:201–3.CrossRefPubMed Manary MP, Boulding W, Staelin R, Glickman SW. The patient experience and health outcomes. N Engl J Med. 2013;368:201–3.CrossRefPubMed
4.
go back to reference Falkenberg K. Why Rating Your Doctor is Bad for Your Health. Forbes. Vol 2015 2013. Falkenberg K. Why Rating Your Doctor is Bad for Your Health. Forbes. Vol 2015 2013.
5.
go back to reference Brown T. Hospitals Aren’t Hotels. The New York Times. March 12, 2012, 2012 Brown T. Hospitals Aren’t Hotels. The New York Times. March 12, 2012, 2012
6.
go back to reference Harris IA, Dao ATT, Young JM, Solomon MJ, Jalaludin BB. Predictors of patient and surgeon satisfaction after orthopaedic trauma. Injury. 2009;40:377–84.CrossRefPubMed Harris IA, Dao ATT, Young JM, Solomon MJ, Jalaludin BB. Predictors of patient and surgeon satisfaction after orthopaedic trauma. Injury. 2009;40:377–84.CrossRefPubMed
7.
go back to reference Lee Hargraves J, Wilson IB, Zaslavsky A, et al. Adjusting for patient characteristics when analyzing reports from patients about hospital care. Med Care. 2001;39:635.CrossRef Lee Hargraves J, Wilson IB, Zaslavsky A, et al. Adjusting for patient characteristics when analyzing reports from patients about hospital care. Med Care. 2001;39:635.CrossRef
8.
go back to reference Young GJ, Meterko M, Desai KR. Patient satisfaction with hospital care: effects of demographic and institutional characteristics. Med Care. 2000;38:325.CrossRefPubMed Young GJ, Meterko M, Desai KR. Patient satisfaction with hospital care: effects of demographic and institutional characteristics. Med Care. 2000;38:325.CrossRefPubMed
9.
go back to reference Elliott MN, Zaslavsky AM, Goldstein E. Effects of Survey Mode, Patient Mix, and Nonresponse on CAHPS® Hospital Survey Scores. Health Serv Res. 2009;44(2 Pt 1):501-18. Elliott MN, Zaslavsky AM, Goldstein E. Effects of Survey Mode, Patient Mix, and Nonresponse on CAHPS® Hospital Survey Scores. Health Serv Res. 2009;44(2 Pt 1):501-18.
10.
go back to reference Elliott MN, Edwards C, Angeles J. Patterns of Unit and Item Nonresponse in the CAHPS® Hospital Survey. Health Serv Res. 2005; 40(6 Pt 2): 2096–2119. Elliott MN, Edwards C, Angeles J. Patterns of Unit and Item Nonresponse in the CAHPS® Hospital Survey. Health Serv Res. 2005; 40(6 Pt 2): 2096–2119.
11.
go back to reference Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the consumer assessment of health plans study survey. Med Care. 2002;40:485.CrossRefPubMed Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the consumer assessment of health plans study survey. Med Care. 2002;40:485.CrossRefPubMed
12.
go back to reference Jha AK, Orav EJ, Zheng J, Epstein AM. Patients' perception of hospital care in the United States. N EnglJ Med. 2008;359:1921–31. Jha AK, Orav EJ, Zheng J, Epstein AM. Patients' perception of hospital care in the United States. N EnglJ Med. 2008;359:1921–31.
13.
go back to reference Graham B, Green A, James M, Katz J, Swiontkowski M. Measuring patient satisfaction in orthopaedic surgery. J Bone Joint Surg Am. 2015;97:80–4.CrossRefPubMed Graham B, Green A, James M, Katz J, Swiontkowski M. Measuring patient satisfaction in orthopaedic surgery. J Bone Joint Surg Am. 2015;97:80–4.CrossRefPubMed
14.
go back to reference Nieman CL, Benke JR, Ishman SL, Smith DF, Boss EF. Whose experience is measured?: a pilot study of patient satisfaction demographics in pediatric otolaryngology. Laryngoscope. 2014;124:290–4.CrossRefPubMed Nieman CL, Benke JR, Ishman SL, Smith DF, Boss EF. Whose experience is measured?: a pilot study of patient satisfaction demographics in pediatric otolaryngology. Laryngoscope. 2014;124:290–4.CrossRefPubMed
15.
go back to reference Vranceanu A-M, Ring D. Factors associated with patient satisfaction. J Hand Surg Am. 2011;36:1504–8.CrossRefPubMed Vranceanu A-M, Ring D. Factors associated with patient satisfaction. J Hand Surg Am. 2011;36:1504–8.CrossRefPubMed
17.
go back to reference Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Q. 2008;72:167–89.CrossRef Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Q. 2008;72:167–89.CrossRef
18.
go back to reference Lewis EF, Hardy M, Snaith B. Estimating the effect of nonresponse bias in a survey of hospital organizations. Eval Health Prof. 2013;36:330–51.CrossRefPubMed Lewis EF, Hardy M, Snaith B. Estimating the effect of nonresponse bias in a survey of hospital organizations. Eval Health Prof. 2013;36:330–51.CrossRefPubMed
19.
go back to reference http://www.pressganey.com/. Press Ganey Corporation, 401 Edgewater Place, Ste 500, Wakefield, MA 01880. Accessed 26 July 2016. http://www.pressganey.com/. Press Ganey Corporation, 401 Edgewater Place, Ste 500, Wakefield, MA 01880. Accessed 26 July 2016.
20.
go back to reference Godil SS, Parker SL, Zuckerman SL, et al. Determining the quality and effectiveness of surgical spine care: patient satisfaction is not a valid p. Spine J. 2013;13:1006–12.CrossRefPubMed Godil SS, Parker SL, Zuckerman SL, et al. Determining the quality and effectiveness of surgical spine care: patient satisfaction is not a valid p. Spine J. 2013;13:1006–12.CrossRefPubMed
21.
go back to reference Anhang Price R, Elliott MN, Zaslavsky AM, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 2014;71(5):522–54.CrossRefPubMedPubMedCentral Anhang Price R, Elliott MN, Zaslavsky AM, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 2014;71(5):522–54.CrossRefPubMedPubMedCentral
22.
go back to reference Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570.CrossRefPubMedPubMedCentral Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570.CrossRefPubMedPubMedCentral
23.
go back to reference Anhang Price R, Elliott MN, Cleary PD, Zaslavsky AM, Hays RD. Should health care providers be accountable for patients’ care experiences? J Gen Intern Med. 2015;30:253–6.CrossRefPubMed Anhang Price R, Elliott MN, Cleary PD, Zaslavsky AM, Hays RD. Should health care providers be accountable for patients’ care experiences? J Gen Intern Med. 2015;30:253–6.CrossRefPubMed
24.
go back to reference Dillman DA, Phelps G, Tortora R, et al. Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc Sci Res. 2009;38(1):1–18.CrossRef Dillman DA, Phelps G, Tortora R, et al. Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc Sci Res. 2009;38(1):1–18.CrossRef
25.
go back to reference Jones R, Pitt N. Health surveys in the workplace: comparison of postal, email and World Wide Web methods. Occup Med. 1999;49(8):556–8.CrossRef Jones R, Pitt N. Health surveys in the workplace: comparison of postal, email and World Wide Web methods. Occup Med. 1999;49(8):556–8.CrossRef
26.
go back to reference Shih T-H, Fan X. Comparing response rates from Web and mail surveys: a meta-analysis. Field Methods. 2008;20(3):249–71.CrossRef Shih T-H, Fan X. Comparing response rates from Web and mail surveys: a meta-analysis. Field Methods. 2008;20(3):249–71.CrossRef
27.
go back to reference Roland M, Elliott M, Lyratzopoulos G, et al. Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England. BMJ. 2009;339:b3851.CrossRefPubMedPubMedCentral Roland M, Elliott M, Lyratzopoulos G, et al. Reliability of patient responses in pay for performance schemes: analysis of national General Practitioner Patient Survey data in England. BMJ. 2009;339:b3851.CrossRefPubMedPubMedCentral
28.
go back to reference Simon G, Rutter C, Crosier M, Scott J. Are comparisons of consumer satisfaction with providers biased by nonresponse or case-mix differences? Psychiatr Serv. 2009;60(1):67-73. Simon G, Rutter C, Crosier M, Scott J. Are comparisons of consumer satisfaction with providers biased by nonresponse or case-mix differences? Psychiatr Serv. 2009;60(1):67-73.
29.
go back to reference Etter JF, Perneger TV. Analysis of non-response bias in a mailed health survey. J Clin Epidemiol. 1997;50(10):1123-8. Etter JF, Perneger TV. Analysis of non-response bias in a mailed health survey. J Clin Epidemiol. 1997;50(10):1123-8.
30.
go back to reference Black N, Varaganum M, Hutchings A. Relationship between patient reported experience (PREMs) and patient reported outcomes (PROMs) in elective surgery. BMJ Qual Saf. 2014;23:534–42.CrossRefPubMed Black N, Varaganum M, Hutchings A. Relationship between patient reported experience (PREMs) and patient reported outcomes (PROMs) in elective surgery. BMJ Qual Saf. 2014;23:534–42.CrossRefPubMed
31.
go back to reference Hamilton DF, Lane JV, Gaston P, et al. What determines patient satisfaction with surgery? A prospective cohort study of 4709 patients following total joint replacement. BMJ Open. 2013;3:e002525.CrossRefPubMedPubMedCentral Hamilton DF, Lane JV, Gaston P, et al. What determines patient satisfaction with surgery? A prospective cohort study of 4709 patients following total joint replacement. BMJ Open. 2013;3:e002525.CrossRefPubMedPubMedCentral
32.
go back to reference Rogers F, Horst M, To T, et al. Factors associated with patient satisfaction scores for physician care in trauma patients. J Trauma Acute Care Surg. 2013;75:110–5.CrossRefPubMed Rogers F, Horst M, To T, et al. Factors associated with patient satisfaction scores for physician care in trauma patients. J Trauma Acute Care Surg. 2013;75:110–5.CrossRefPubMed
33.
go back to reference Abtahi AM, Presson AP, Zhang C, Saltzman CL, Tyser AR. Association between orthopaedic outpatient satisfaction and non-modifiable patient factors. J Bone Joint Surg Am. 2015;97(13):1041–8.CrossRefPubMedPubMedCentral Abtahi AM, Presson AP, Zhang C, Saltzman CL, Tyser AR. Association between orthopaedic outpatient satisfaction and non-modifiable patient factors. J Bone Joint Surg Am. 2015;97(13):1041–8.CrossRefPubMedPubMedCentral
34.
go back to reference Molinari NM, Wolter KM, Skalland B, et al. Quantifying bias in a health survey: modeling total survey error in the national immunization survey. Stat Med. 2011;30(5):505–14.PubMed Molinari NM, Wolter KM, Skalland B, et al. Quantifying bias in a health survey: modeling total survey error in the national immunization survey. Stat Med. 2011;30(5):505–14.PubMed
35.
go back to reference Drake KM, Hargraves JL, Lloyd S, Gallagher PM, Cleary PD. The effect of response scale, administration mode, and format on responses to the CAHPS clinician and group survey. Health Serv Res. 2014;49:1387–99.CrossRefPubMedPubMedCentral Drake KM, Hargraves JL, Lloyd S, Gallagher PM, Cleary PD. The effect of response scale, administration mode, and format on responses to the CAHPS clinician and group survey. Health Serv Res. 2014;49:1387–99.CrossRefPubMedPubMedCentral
36.
go back to reference Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805–6.CrossRefPubMed Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805–6.CrossRefPubMed
Metadata
Title
Evidence of non-response bias in the Press-Ganey patient satisfaction survey
Authors
A. R. Tyser
A. M. Abtahi
M. McFadden
A. P. Presson
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2016
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/s12913-016-1595-z

Other articles of this Issue 1/2016

BMC Health Services Research 1/2016 Go to the issue