Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2018

Open Access 01-12-2018 | Research article

A randomized comparison of three data collection models for the measurement of parent experiences with diabetes outpatient care

Authors: Oyvind Bjertnaes, Hilde Hestad Iversen, Torild Skrivarhaug

Published in: BMC Medical Research Methodology | Issue 1/2018

Login to get access

Abstract

Background

The objective of this study was to compare three data collection methods for the measurement of parent experiences with hospital outpatient care for child and adolescent diabetes, based on a randomised national trial in Norway involving both pen-and-paper and electronic response options.

Methods

The sample frame was patients registered in the Norwegian Childhood Diabetes Registry. Parents of patients were randomised into the following groups (n = 2606): group A, who were posted questionnaires with only a pen-and-paper response option (n = 859); group B, who were posted questionnaires with both an electronic and a pen-and-paper response option (n = 886); and group C, who were posted questionnaires with only an electronic response option (n = 861). The three groups were compared on response rate, background variables about respondents, main study results and survey costs. Statistical analysis included logistic regression to test group differences in response probabilities and multilevel linear regression to test differences in parent experiences.

Results

The response rate was 61.8% for group A, 62.4% for group B and 41.6% for group C. The probability of answering was significantly higher for group A (OR = 2.3, p < 0.001) and B (OR = 2.3, p < 0.001) compared to group C. Respondent age, gender, education, living with the child and the degree of participation in consultations did not differ significantly between the three groups. Group differences in parent-reported experiences were small, varying from 1.0 (equipment and doctor contact) to 2.4 (outcome), on a scale from 0 to 100. Only one of 18 group differences was significant: the mixed group had significantly higher score than the electronic group on the organization scale (p < 0.05). The total cost of the electronic model was less than half the cost of the other models, and cost per response was 5.1 euros for the electronic model compared to 8.2 euros for group A and 7.6 euros for group B.

Conclusions

The models with pen-and paper questionnaire included had more than 20% higher response rate than the model with an electronic-only response option. Background variables and parent-reported experiences were similar between the three groups, and the electronic model was the more cost-effective model.
Appendix
Available only for authorised users
Literature
1.
go back to reference Beattie M, Murphy DJ, Atherton I, Lauder W. Instruments to measure patient experience of healthcare quality in hospitals: a systematic review. Syst Rev. 2015;4:97.CrossRef Beattie M, Murphy DJ, Atherton I, Lauder W. Instruments to measure patient experience of healthcare quality in hospitals: a systematic review. Syst Rev. 2015;4:97.CrossRef
2.
go back to reference Castle NG, Brown J, Hepner KA, Hays RD. Review of the literature on survey instruments used to collect data on hospital patients’ perceptions of care. Health Serv Res. 2005;40:1996–2017.CrossRef Castle NG, Brown J, Hepner KA, Hays RD. Review of the literature on survey instruments used to collect data on hospital patients’ perceptions of care. Health Serv Res. 2005;40:1996–2017.CrossRef
3.
go back to reference Bergeson SC, Gray J, Ehrmantraut LA, Laibson T, Hays RD. Comparing web-based with mail survey administration of the consumer assessment of healthcare providers and systems (CAHPS(®)) clinician and group survey. Prim Health Care. 2013;3:1–15. Bergeson SC, Gray J, Ehrmantraut LA, Laibson T, Hays RD. Comparing web-based with mail survey administration of the consumer assessment of healthcare providers and systems (CAHPS(®)) clinician and group survey. Prim Health Care. 2013;3:1–15.
4.
go back to reference Bjertnaes OA, Iversen HH. User-experience surveys with maternity services: a randomized comparison of two data collection models. Int J Qual Health Care. 2012;24:433–8.CrossRef Bjertnaes OA, Iversen HH. User-experience surveys with maternity services: a randomized comparison of two data collection models. Int J Qual Health Care. 2012;24:433–8.CrossRef
5.
go back to reference Zuidgeest M, Hendriks M, Koopman L, Spreeuwenberg P, Rademakers J. A comparison of a postal survey and mixed-mode survey using a questionnaire on patients' experiences with breast care. J Med Internet Res. 2011;13:e68.CrossRef Zuidgeest M, Hendriks M, Koopman L, Spreeuwenberg P, Rademakers J. A comparison of a postal survey and mixed-mode survey using a questionnaire on patients' experiences with breast care. J Med Internet Res. 2011;13:e68.CrossRef
6.
go back to reference Schleyer TK, Forrest JL. Methods for the design and administration of web-based surveys. J Am Med Inform Assoc. 2000;7:416–25.CrossRef Schleyer TK, Forrest JL. Methods for the design and administration of web-based surveys. J Am Med Inform Assoc. 2000;7:416–25.CrossRef
7.
go back to reference Millar MM, O’Neill AC, Dillman DA. Are mode preferences real? Washington State University. In: SESRC 2009, technical report 09–003. Millar MM, O’Neill AC, Dillman DA. Are mode preferences real? Washington State University. In: SESRC 2009, technical report 09–003.
8.
go back to reference Gleeson H, Calderon A, Swami V, Deighton J, Wolpert M, Edbrooke-Childs J. Systematic review of approaches to using patient experience data for quality improvement in healthcare settings. BMJ Open. 2016;6:e011907.CrossRef Gleeson H, Calderon A, Swami V, Deighton J, Wolpert M, Edbrooke-Childs J. Systematic review of approaches to using patient experience data for quality improvement in healthcare settings. BMJ Open. 2016;6:e011907.CrossRef
9.
go back to reference Elliott MN, Brown JA, Lehrman WG, Beckett MK, Hambarsoomian K, Giordano LA, et al. A randomized experiment investigating the suitability of speech-enabled IVR and web modes for publicly reported surveys of patients' experience of hospital care. Med Care Res Rev. 2013;70:165–84.CrossRef Elliott MN, Brown JA, Lehrman WG, Beckett MK, Hambarsoomian K, Giordano LA, et al. A randomized experiment investigating the suitability of speech-enabled IVR and web modes for publicly reported surveys of patients' experience of hospital care. Med Care Res Rev. 2013;70:165–84.CrossRef
10.
go back to reference Rodriguez HP, von Glahn T, Rogers WH, Chang H, Fanjiang G, Safran DG. Evaluating patients' experiences with individual physicians: a randomized trial of mail, internet, and interactive voice response telephone administration of surveys. Med Care. 2006;44:167–74.CrossRef Rodriguez HP, von Glahn T, Rogers WH, Chang H, Fanjiang G, Safran DG. Evaluating patients' experiences with individual physicians: a randomized trial of mail, internet, and interactive voice response telephone administration of surveys. Med Care. 2006;44:167–74.CrossRef
11.
go back to reference Shih TH, Xitao F. Response rates and mode preferences in web-mail mixed-mode surveys: a meta-analysis. Int J Internet Sci. 2007;2:59–82. Shih TH, Xitao F. Response rates and mode preferences in web-mail mixed-mode surveys: a meta-analysis. Int J Internet Sci. 2007;2:59–82.
12.
go back to reference Shih TH, Fan X. Comparing response rates from web and mail surveys: a meta-analysis. Field Methods. 2008;20:249–71.CrossRef Shih TH, Fan X. Comparing response rates from web and mail surveys: a meta-analysis. Field Methods. 2008;20:249–71.CrossRef
13.
go back to reference Couturier B, Carrat F, Hejblum G, SENTIPAT Study Group. Comparing patients’ opinions on the hospital discharge process collected with a self-reported questionnaire completed via the internet or through a telephone survey: An Ancillary Study of the SENTIPAT Randomized Controlled Trial. J Med Internet Res. 2015;17:e158.CrossRef Couturier B, Carrat F, Hejblum G, SENTIPAT Study Group. Comparing patients’ opinions on the hospital discharge process collected with a self-reported questionnaire completed via the internet or through a telephone survey: An Ancillary Study of the SENTIPAT Randomized Controlled Trial. J Med Internet Res. 2015;17:e158.CrossRef
14.
go back to reference Holmboe O, Bjertnæs OA. Patient experiences with Norwegian hospitals in 2015. Results following a national survey. Oslo: Norwegian Knowledge Centre for the Health Services, PasOpp-report 147; 2016. ISSN 1890-1565. Holmboe O, Bjertnæs OA. Patient experiences with Norwegian hospitals in 2015. Results following a national survey. Oslo: Norwegian Knowledge Centre for the Health Services, PasOpp-report 147; 2016. ISSN 1890-1565.
15.
go back to reference Holmboe O, Danielsen K, Iversen HH. Development of a method for the measurement of patient experiences with regular general practitioners. Oslo: Norwegian Knowledge Centre for the Health Services, PasOpp-report 1; 2015. ISSN 1890-1565. Holmboe O, Danielsen K, Iversen HH. Development of a method for the measurement of patient experiences with regular general practitioners. Oslo: Norwegian Knowledge Centre for the Health Services, PasOpp-report 1; 2015. ISSN 1890-1565.
16.
go back to reference Danielsen K, Holmboe O, Hansen SL, Førland O, Bjertnæs ØA. Patient experiences with Norwegian primary care out-of-hours clinics. Oslo: Results from a survey among the “Vakttårn” clinics. Norwegian Knowledge Centre for the Health Services, PasOpp-report 1; 2012. ISSN 1890-1565. Danielsen K, Holmboe O, Hansen SL, Førland O, Bjertnæs ØA. Patient experiences with Norwegian primary care out-of-hours clinics. Oslo: Results from a survey among the “Vakttårn” clinics. Norwegian Knowledge Centre for the Health Services, PasOpp-report 1; 2012. ISSN 1890-1565.
17.
go back to reference Elliott MN, Zaslavsky AM, Goldstein E, Lehrman W, Hambarsoomians K, Beckett MK, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44:501–18.CrossRef Elliott MN, Zaslavsky AM, Goldstein E, Lehrman W, Hambarsoomians K, Beckett MK, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44:501–18.CrossRef
18.
go back to reference Edwards PJ, Roberts I, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;3:MR000008. Edwards PJ, Roberts I, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;3:MR000008.
19.
go back to reference Haugum M, Danielsen K, Iversen HH, Bjertnaes O. The use of data from national and other large-scale user experience surveys in local quality work: a systematic review. Int J Qual Health Care. 2014;26:592–605.CrossRef Haugum M, Danielsen K, Iversen HH, Bjertnaes O. The use of data from national and other large-scale user experience surveys in local quality work: a systematic review. Int J Qual Health Care. 2014;26:592–605.CrossRef
20.
go back to reference Halbesleben JR, Whitman MV. Evaluating survey quality in health services research: a decision framework for assessing nonresponse bias. Health Serv Res. 2013;48:913–30.CrossRef Halbesleben JR, Whitman MV. Evaluating survey quality in health services research: a decision framework for assessing nonresponse bias. Health Serv Res. 2013;48:913–30.CrossRef
21.
go back to reference Garratt AM, Bjertnaes OA, Holmboe O, Hanssen-Bauer K. Parent experiences questionnaire for outpatient child and adolescent mental health services (PEQ-CAMHS outpatients): reliability and validity following a national survey. Child Adolesc Psychiatry Ment Health. 2011;5:18.CrossRef Garratt AM, Bjertnaes OA, Holmboe O, Hanssen-Bauer K. Parent experiences questionnaire for outpatient child and adolescent mental health services (PEQ-CAMHS outpatients): reliability and validity following a national survey. Child Adolesc Psychiatry Ment Health. 2011;5:18.CrossRef
22.
go back to reference Bjertnaes OA, Garratt A, Iversen H, Ruud T. The association between GP and patient ratings of quality of care at outpatient clinics. Fam Pract. 2009;26:384–90.CrossRef Bjertnaes OA, Garratt A, Iversen H, Ruud T. The association between GP and patient ratings of quality of care at outpatient clinics. Fam Pract. 2009;26:384–90.CrossRef
Metadata
Title
A randomized comparison of three data collection models for the measurement of parent experiences with diabetes outpatient care
Authors
Oyvind Bjertnaes
Hilde Hestad Iversen
Torild Skrivarhaug
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2018
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-018-0557-z

Other articles of this Issue 1/2018

BMC Medical Research Methodology 1/2018 Go to the issue