Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Emerging Themes in Epidemiology
Published in: Emerging Themes in Epidemiology 1/2017

Open Access 01-12-2017 | Research article

Comparison of response patterns in different survey designs: a longitudinal panel with mixed-mode and online-only design

Authors: Nicole Rübsamen, Manas K. Akmatov, Stefanie Castell, André Karch, Rafael T. Mikolajczyk

Published in: Emerging Themes in Epidemiology | Issue 1/2017

Login to get access

Abstract

Background

Increasing availability of the Internet allows using only online data collection for more epidemiological studies. We compare response patterns in a population-based health survey using two survey designs: mixed-mode (choice between paper-and-pencil and online questionnaires) and online-only design (without choice).

Methods

We used data from a longitudinal panel, the Hygiene and Behaviour Infectious Diseases Study (HaBIDS), conducted in 2014/2015 in four regions in Lower Saxony, Germany. Individuals were recruited using address-based probability sampling. In two regions, individuals could choose between paper-and-pencil and online questionnaires. In the other two regions, individuals were offered online-only participation. We compared sociodemographic characteristics of respondents who filled in all panel questionnaires between the mixed-mode group (n = 1110) and the online-only group (n = 482). Using 134 items, we performed multinomial logistic regression to compare responses between survey designs in terms of type (missing, “do not know” or valid response) and ordinal regression to compare responses in terms of content. We applied the false discovery rates (FDR) to control for multiple testing and investigated effects of adjusting for sociodemographic characteristic. For validation of the differential response patterns between mixed-mode and online-only, we compared the response patterns between paper and online mode among the respondents in the mixed-mode group in one region (n = 786).

Results

Respondents in the online-only group were older than those in the mixed-mode group, but both groups did not differ regarding sex or education. Type of response did not differ between the online-only and the mixed-mode group. Survey design was associated with different content of response in 18 of the 134 investigated items; which decreased to 11 after adjusting for sociodemographic variables. In the validation within the mixed-mode, only two of those were among the 11 significantly different items. The probability of observing by chance the same two or more significant differences in this setting was 22%.

Conclusions

We found similar response patterns in both survey designs with only few items being answered differently, likely attributable to chance. Our study supports the equivalence of the compared survey designs and suggests that, in the studied setting, using online-only design does not cause strong distortion of the results.
Appendix
Available only for authorised users
Literature
1.
Smith B, Smith TC, Gray GC, Ryan MAK. When epidemiology meets the Internet: web-based surveys in the Millennium Cohort Study. Am J Epidemiol. 2007;166:1345–54.CrossRefPubMed
2.
Owen JE, Bantum EOC, Criswell K, Bazzo J, Gorlick A, Stanton AL. Representativeness of two sampling procedures for an internet intervention targeting cancer-related distress: a comparison of convenience and registry samples. J Behav Med. 2014;37:630–41.CrossRefPubMed
3.
Özdemir RS, St. Louis KO, Topbaş S. Public attitudes toward stuttering in Turkey: probability versus convenience sampling. J Fluency Disord. 2011;36:262–7.CrossRefPubMed
4.
Hultsch DF, MacDonald SWS, Hunter MA, Maitland SB, Dixon RA. Sampling and generalisability in developmental research: comparison of random and convenience samples of older adults. Int J Behav Dev. 2002;26:345–59.CrossRef
5.
Greenlaw C, Brown-Welty S. A comparison of web-based and paper-based survey methods: testing assumptions of survey mode and response cost. Eval Rev. 2009;33:464–80.CrossRefPubMed
6.
Hohwü L, Lyshol H, Gissler M, Jonsson SH, Petzold M, Obel C. Web-based versus traditional paper questionnaires: a mixed-mode survey with a Nordic perspective. J Med Internet Res. 2013;15:e173.CrossRefPubMedPubMedCentral
7.
Kypri K, Samaranayaka A, Connor J, Langley JD, Maclennan B. Non-response bias in a web-based health behaviour survey of New Zealand tertiary students. Prev Med (Baltim) Elsevier Inc. 2011;53:274–7.CrossRef
8.
McCluskey S, Topping AE. Increasing response rates to lifestyle surveys: a pragmatic evidence review. Perspect Public Health. 2011;131:89–94.CrossRefPubMed
9.
Plante C, Jacques L, Chevalier S, Fournier M. Comparability of internet and telephone data in a survey on the respiratory health of children. Can Respir J. 2012;19:13–8.CrossRefPubMedPubMedCentral
10.
Shim J-M, Shin E, Johnson TP. Self-rated health assessed by web versus mail modes in a mixed mode survey. Med Care. 2013;51:774–81.CrossRefPubMed
11.
Callas PW, Solomon LJ, Hughes JR, Livingston AE. The influence of response mode on study results: offering cigarette smokers a choice of postal or online completion of a survey. J Med Internet Res. 2010;12:e46.CrossRefPubMedPubMedCentral
12.
McCabe SE, Diez A, Boyd CJ, Nelson TF, Weitzman ER. Comparing web and mail responses in a mixed mode survey in college alcohol use research. Addict Behav. 2006;31:1619–27.CrossRefPubMedPubMedCentral
13.
Christensen AI, Ekholm O, Glümer C, Juel K. Effect of survey mode on response patterns: comparison of face-to-face and self-administered modes in health surveys. Eur J Public Health. 2014;24:327–32.CrossRefPubMed
14.
Schilling R, Hoebel J, Müters S, Lange C. Pilot study on the implementation of mixed-mode health interview surveys in the adult population (GEDA 2.0). Beiträge zur Gesundheitsberichterstattung des Bundes. 2015 (in German).
15.
Sinclair M, O’Toole J, Malawaraarachchi M, Leder K. Comparison of response rates and cost-effectiveness for a community-based survey: postal, internet and telephone modes with generic or personalised recruitment approaches. BMC Med Res Methodol. 2012;12:132.CrossRefPubMedPubMedCentral
16.
Rübsamen N, Akmatov M, Karch A, Schweitzer A, Zoch B, Mikolajczyk RT. The HaBIDS study (Hygiene and Behaviour Infectious Diseases Study)—setting up an online panel in Lower Saxony. Abstracts for the 9th Annual Meeting of the German Society for Epidemiology, Ulm; 2014 Sep 17–20: German Society of Epidemiology (DGEpi); 2014. p. Abstract P27 (in German).
17.
18.
Hoffmeyer-Zlotnik JHP, Glemser A, Heckel C, von der Heyde C, Quitt H, Hanefeld U, et al. Demographic standards. Stat. und Wiss. 5th ed. Wiesbaden: Federal Statistical Office; 2010 (in German).
19.
Thefeld W, Stolzenberg H, Bellach BM. The Federal Health Survey: response, composition of participants and non-responder analysis. Gesundheitswesen. 1999;61 Spec No:S57-61 (in German).
20.
Sievers C, Akmatov MK, Kreienbrock L, Hille K, Ahrens W, Günther K, et al. Evaluation of a questionnaire to assess selected infectious diseases and their risk factors. Bundesgesundheitsblatt. 2014;57:1283–91.CrossRef
21.
Mossong J, Hens N, Jit M, Beutels P, Auranen K, Mikolajczyk R, et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med. 2008;5:e74.CrossRefPubMedPubMedCentral
22.
Mercer CH, Tanton C, Prah P, Erens B, Sonnenberg P, Clifton S, et al. Changes in sexual attitudes and lifestyles in Britain through the life course and over time: findings from the National Surveys of Sexual Attitudes and Lifestyles (Natsal). Lancet. 2013;382:1781–94 (Mercer et al. Open Access article distributed under the terms of CC BY).CrossRefPubMedPubMedCentral
23.
Valente TW, Paredes P, Poppe PR. Matching the message to the process the relative ordering of knowledge, attitudes, and practices in behavior change research. Hum Commun Res. 1998;24:366–85.CrossRefPubMed
24.
Cardiff Software Inc. TELEform software. California City; 2002.
25.
26.
Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B. 1995;57:289–300.
27.
Verhoeven KJF, Simonsen KL, McIntyre LM. Implementing false discovery rate control: increasing your power. Oikos. 2005;108:643–7.CrossRef
28.
StataCorp. Stata statistical software: release 12. College Station: StataCorp LP.; 2011.
29.
The American Association for Public Opinion Research. Standard definitions: final dispositions of case codes and outcome rates for surveys, 8th edn. AAPOR; 2015.
30.
31.
32.
Long JS, Freese J. Regression models for categorical dependent variables using Stata. 3rd ed. College Station: Stata Press; 2014.
33.
Royston P, Ambler G, Sauerbrei W. The use of fractional polynomials to model continuous risk variables in epidemiology. Int J Epidemiol. 1999;28:964–74.CrossRefPubMed
34.
Manfreda KL, Bosnjak M, Berzelak J, Haas I, Vehovar V. Web surveys versus other survey modes: a meta-analysis comparing response rates. Int J Mark Res. 2008;50:79–104.
35.
Eagan TML, Eide GE, Gulsvik A, Bakke PS. Nonresponse in a community cohort study: predictors and consequences for exposure-disease associations. J Clin Epidemiol. 2002;55:775–81.CrossRefPubMed
36.
Dunn KM, Jordan K, Lacey RJ, Shapley M, Jinks C. Patterns of consent in epidemiologic research: evidence from over 25,000 responders. Am J Epidemiol. 2004;159:1087–94.CrossRefPubMed
37.
Keiding N, Louis TA. Perils and potentials of self-selected entry to epidemiological studies and surveys. J R Stat Soc A. 2016;179:1–28.CrossRef
38.
Hoffmann W, Terschüren C, Holle R, Kamtsiuris P, Bergmann M, Kroke A, et al. The problem of response in epidemiological studies in Germany (part II). Gesundheitswesen. 2004;66:482–91.CrossRefPubMed
39.
Stang A, Jöckel KH. Studies with low response proportions may be less biased than studies with high response proportions. Am J Epidemiol. 2004;159:204–10.CrossRefPubMed
40.
Nohr EA, Frydenberg M, Henriksen TB, Olsen J. Does low participation in cohort studies induce bias? Epidemiology. 2006;17:413–8.CrossRefPubMed
Metadata
Title
Comparison of response patterns in different survey designs: a longitudinal panel with mixed-mode and online-only design
Authors
Nicole Rübsamen
Manas K. Akmatov
Stefanie Castell
André Karch
Rafael T. Mikolajczyk
Publication date
01-12-2017
Publisher
BioMed Central
Published in
Emerging Themes in Epidemiology / Issue 1/2017
Electronic ISSN: 1742-7622
DOI
https://doi.org/10.1186/s12982-017-0058-2

Other articles of this Issue 1/2017

Emerging Themes in Epidemiology 1/2017 Go to the issue

Research article

Flexible semiparametric joint modeling: an application to estimate individual lung function decline and risk of pulmonary exacerbations in cystic fibrosis

Review

Participatory epidemiology: the contribution of participatory research to epidemiology

Review

Spatial analysis of cluster randomised trials: a systematic review of analysis methods

Research article

Multiple imputation using linked proxy outcome data resulted in important bias reduction and efficiency gains: a simulation study

Analytic perspective

Randomised and non-randomised studies to estimate the effect of community-level public health interventions: definitions and methodological considerations

Analytic perspective

On hazard ratio estimators by proportional hazards models in matched-pair cohort studies