Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Research article

Mixing mixed-mode designs in a national health interview survey: a pilot study to assess the impact on the self-administered questionnaire non-response

Authors: Elise Braekman, Sabine Drieskens, Rana Charafeddine, Stefaan Demarest, Finaba Berete, Lydia Gisle, Jean Tafforeau, Johan Van der Heyden, Guido Van Hal

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

Many population health surveys consist of a mixed-mode design that includes a face-to-face (F2F) interview followed by a paper-and-pencil (P&P) self-administered questionnaire (SAQ) for the sensitive topics. In order to alleviate the burden of a supplementary P&P questioning after the interview, a mixed-mode SAQ design including a web and P&P option was tested for the Belgian health interview survey.

Methods

A pilot study (n = 266, age 15+) was organized using a mixed-mode SAQ design following the F2F interview. Respondents were invited to complete a web SAQ either immediately after the interview or at a later time. The P&P option was offered in case respondents refused or had previously declared having no computer access, no internet connection or no recent usage of computers. The unit response rate for the web SAQ and the overall unit response rate for the SAQ independent of the mode were evaluated. A logistic regression analysis was conducted to explore the association of socio-demographic characteristics and interviewer effects with the completed SAQ mode. Furthermore, a logistic regression analysis assessed the differential user-friendliness of the SAQ modes. Finally, a logistic multilevel model was used to evaluate the item non-response in the two SAQ modes while controlling for respondents’ characteristics.

Results

Of the eligible F2F respondents in this study, 76% (107/140) agreed to complete the web SAQ. Yet among those, only 78.5% (84/107) actually did. At the end, the overall (web and P&P) SAQ unit response rate reached 73.5%. In this study older people were less likely to complete the web SAQ. Indications for an interviewer effect were observed as regard the number of web respondents, P&P respondents and respondents who refused to complete the SAQ. The web SAQ scored better in terms of user-friendliness and presented higher item response than the P&P SAQ.

Conclusions

The web SAQ performed better regarding user-friendliness and item response than the P&P SAQ but the overall SAQ unit response rate was low. Therefore, future research is recommended to further assess which type of SAQ design implemented after a F2F interview is the most beneficial to obtain high unit and item response rates.
Literature
1.
Demarest S, Van der Heyden J, Charafeddine R, Drieskens S, Gisle L, Tafforeau J. Methodological basics and evolution of the Belgian health interview survey 1997-2008. Arch Public Health. 2013;71:24.CrossRef
2.
Ekholm O, Hesse U, Davidsen M, Kjoller M. The study design and characteristics of the Danish national health interview surveys. Scand J Public Health. 2009;37:758–65.CrossRef
3.
Mindell JS, Tipping S, Pickering K, Hope S, Roth MA, Erens B. The effect of survey method on survey participation: analysis of data from the health survey for England 2006 and the boost survey for London. BMC Med Res Methodol. 2010;10:83.CrossRef
4.
5.
De Leeuw ED, Hox JJ. Internet surveys as part of a mixed-mode design. In: Das M, Ester P, Kaczmirek L, editors. Social and behavioral research and the Internet: Advances in applied methods and new research strategies. New York: Routledge, Taylor and Francis Group; 2011. p. 45–76.
6.
Berete F, Van der Heyden J, Demarest S, Charafeddine R, Gisle L, Braekman E, et al. Determinants of unit nonresponse in multi-mode data collection: a multilevel analysis. PLoS One. 2019;14(4):e0215652.CrossRef
7.
Groves RM, Peytcheva E. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opin Q. 2008;72:167–89.CrossRef
8.
Daikeler J, Bošnjak M, Manfreda KL. Web versus other survey modes: an updated and extended meta-analysis comparing response rates. J Surv Stat Methodol. 2019;0:1–27.
9.
Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17:643–53.CrossRef
10.
Gnambs T, Kaspar K. Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis. Behav Res Methods. 2015;47:1237–59.CrossRef
11.
Touvier M, Méjean C, Kesse-Guyot E, Pollet C, Malon A, Castetbon K, et al. Comparison between web-based and paper versions of a self-administered anthropometric questionnaire. Eur J Epidemiol. 2010;25:287–96.CrossRef
12.
Vergnaud AC, Touvier M, Méjean C, Kesse-Guyot E, Pollet C, Malon A, et al. Agreement between web-based and paper versions of a socio-demographic questionnaire in the NutriNet-Santé study. Int J Public Health. 2011;56:407–17.CrossRef
13.
Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health. 2005;27:281–91.CrossRef
14.
Kwak N, Radler B. A comparison between mail and web surveys: response pattern, respondent profile, and data quality. J Off Stat. 2002;18:257–73.
15.
Mauz E, von der Lippe E, Allen J, Schilling R, Müters S, Hoebel J, et al. Mixing modes in a population-based interview survey: comparison of a sequential and a concurrent mixed-mode design for public health research. Arch Public Health. 2018;76:8.
16.
Krosnick JA. Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cogn Psychol. 1991;5:213–36.CrossRef
17.
Fang J, Prybutok V, Wen C. Shirking behavior and socially desirable responding in online surveys: a cross-cultural study comparing Chinese and American samples. Comput Hum Behav. 2016;54:310–7.CrossRef
18.
Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008;11:322–33.CrossRef
19.
Shim JM, Shin E, Johnson TP. Self-rated health assessed by web versus mail modes in a mixed mode survey: the digital divide effect and the genuine survey mode effect. Med Care. 2013;51:774–81.CrossRef
20.
Bethlehem J. Web surveys in official statistics. In: Engel U, Jann B, Lynn P, Scherpenzeel A, Sturgis P, editors. Improving survey methods: lessons from recent research. New York: Routledge; 2015. p. 156–69.
21.
Millar MM, Dillman DA. Improving response to web and mixed-mode surveys. Public Opin Q. 2011;75:249–69.CrossRef
22.
23.
24.
Rosell-Murphy M, Rodriguez-Blanco T, Morán J, Pons-Vigués M, Elorza-Ricart JM, Rodríguez J, et al. Variability in screening prevention activities in primary care in Spain: a multilevel analysis. BMC Public Health. 2015;15:473.CrossRef
25.
Merlo J, Chaix B, Ohlsson H, Beckman A, Johnell K, Hjerpe P, et al. A brief conceptual tutorial of multilevel analysis in social epidemiology: using measures of clustering in multilevel logistic regression to investigate contextual phenomena. J Epidemiol Community Health. 2006;60:290–7.CrossRef
26.
McCluskey S, Topping AE. Increasing response rates to lifestyle surveys: a pragmatic evidence review. Perspect Public Health. 2011;131:89–94.CrossRef
27.
Braekman E, Charafeddine R, Demarest S, Drieskens S, Tafforeau J, Van der Heyden J, et al. Is the European Health Interview Survey online yet? Response and net sample composition of a web-based data collection. Eur J Public Health. 2019;ckz206. https://​doi.​org/​10.​1093/​eurpub/​ckz206.
28.
Jensen HAR, Ekholm O, Davidsen M, Christensen AI. The Danish health and morbidity surveys: study design and participant characteristics. BMC Med Res Methodol. 2019;19:91.CrossRef
29.
Volken T. Second-stage non-response in the Swiss health survey: determinants and bias in outcomes. BMC Public Health. 2013;13:167.CrossRef
30.
Akmatov MK, Rübsamen N, Schultze A, Kemmling Y, Obi N, Günther K, et al. Diverse recruitment strategies result in different participation percentages in a web-based study, but in similar compliance. Int J Public Health. 2015;60:937–43.CrossRef
31.
Messer BL, Edwards ML, Dillman DA. Determinants of item nonresponse to web and mail respondents in three address-based mixed-mode surveys of the general public. Surv Pract. 2012;5:1–8.CrossRef
32.
Dillman DA. The promise and challenge of pushing respondents to the web in mixed-mode surveys. Surv Methodol. 2017;43:3–30.
33.
Morton SMB, Bandara DK, Robinson EM, Carr PEA. In the 21st century, what is an acceptable response rate? Aust N Z J Public Health. 2012;36:106–8.CrossRef
34.
Zuidgeest M, Hendriks M, Koopman L, Spreeuwenberg P, Rademakers J. A comparison of a postal survey and mixed-mode survey using a questionnaire on patients’ experiences with breast care. J Med Internet Res. 2011;13(3):e68.CrossRef
35.
Mohorko A, De Leeuw ED, Hox J. Internet coverage and coverage bias in Europe: developments across countries and over time. J Off Stat. 2013;29:609–22.CrossRef
36.
Leenheer J, Scherpenzeel AC. Does it pay off to include non-internet households in an internet panel? Int J Internet Sci. 2013;8:17–29.
37.
Sterrett D, Malato D, Benz J, Tompson T, English N. Assessing changes in coverage bias of web surveys in the United States. Public Opin Q. 2017;81:338–56.CrossRef
38.
39.
Ackermann-Piek D, Massing N. Interviewer behavior and interviewer characteristics in PIAAC Germany. Methods Data Analyses. 2014;8:199–222.
40.
Naus MJ, Philipp LM, Samsi M. From paper to pixels: a comparison of paper and computer formats in psychological assessment. Comput Hum Behav. 2009;25:1–7.CrossRef
41.
Galesic M. Dropouts on the web: effects of interest and burden experienced during an online survey. J Off Stat. 2006;22:313–28.
42.
Borgers N, Hox J. Item nonresponse in questionnaire research with children. J Off Stat. 2001;17:321–35.
Metadata
Title
Mixing mixed-mode designs in a national health interview survey: a pilot study to assess the impact on the self-administered questionnaire non-response
Authors
Elise Braekman
Sabine Drieskens
Rana Charafeddine
Stefaan Demarest
Finaba Berete
Lydia Gisle
Jean Tafforeau
Johan Van der Heyden
Guido Van Hal
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0860-3

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue

Research article

Comparison of a time-varying covariate model and a joint model of time-to-event outcomes in the presence of measurement error and interval censoring: application to kidney transplantation

Research article

The power of the group: comparison of interviews and group concept mapping for identifying patient-important outcomes of care

Research article

The relationship between statistical power and predictor distribution in multilevel logistic regression: a simulation-based approach

Research article

The impact of varying cluster size in cross-sectional stepped-wedge cluster randomised trials

Research article

Dynamic prediction of repeated events data based on landmarking model: application to colorectal liver metastases data

Research article

Strategies for recruitment and retention of underrepresented populations with chronic obstructive pulmonary disease for a clinical trial