Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2020

Open Access 01-12-2020 | Tick | Research article

Telephone versus web panel National Survey for monitoring adoption of preventive behaviors to climate change in populations: a case study of Lyme disease in Québec, Canada

Authors: Grâce Ngambo Domche, Pierre Valois, Magalie Canuel, Denis Talbot, Maxime Tessier, Cécile Aenishaenslin, Catherine Bouchard, Sandie Briand

Published in: BMC Medical Research Methodology | Issue 1/2020

Login to get access

Abstract

Background

To monitor the adoption of climate change adaptive behaviors in the population, public health authorities have to conduct national surveys, which can help them target vulnerable subpopulations. To ensure reliable estimates of the adoption of these preventive behaviors, many data collection methods are offered by polling firms. The aim of this study was to compare a telephone survey with a web survey on Lyme disease with regard to their representativeness.

Methods

The data comes from a cross-sectional study conducted in the Province of Québec (Canada). In total, 1003 people completed the questionnaire by telephone and 956 filled in a web questionnaire. We compared the data obtained from both survey modes with the census data in regard to various demographic characteristics. We then compared the data from both samples in terms of self-reported Lyme disease preventive behaviors and other theoretically associated constructs. We also assessed the measurement invariance (equivalence) of the index of Lyme disease preventive behaviors across the telephone and web samples.

Results

Findings showed that neither the telephone nor the web panel modes of data collection can be considered more representative of the target population. The results showed that the proportion of item non-responses was significantly higher with the web questionnaire (5.6%) than with the telephone survey (1.3%), and that the magnitude of the differences between the two survey modes was nil for 19 out of the 30 items related to Lyme disease, and small for 11 of them. Results from invariance analyses confirmed the measurement invariance of an index of adaptation to Lyme disease, as well as the mean invariance across both samples.

Conclusions

Our results suggested that both samples provided similar estimates of the level of adaptation to Lyme disease preventive behaviors. In sum, the results of our study showed that neither survey mode was superior to the other. Thus, in studies where adaptation to climate change is monitored over time, using a web survey instead of a telephone survey could be more cost-effective, and researchers should consider doing so in future surveys on adaptation to climate. However, we recommend conducting a pretest study before deciding whether to use both survey modes or only one of them.
Appendix
Available only for authorised users
Literature
1.
Masson-Delmotte V, Zhai P, Pörtner HO, Roberts D, Skea J, Shukla PR, et al. Global warming of 1.5 C An IPCC Special Report on the impacts of global warming of 1.5 C above pre-industrial levels and related global greenhouse gas emission pathways, in the context of strengthening the global response to the threat of climate change, sustainable development, and efforts to eradicate poverty. Summary for Policymakers Edited by Science Officer Science Assistant, 2018.
2.
Desjarlais C, Blondlot A. Savoir s’adapter aux changements climatiques. Montréal: Ouranos; 2010. p. 128.
3.
Health Canada. You’re active in the heat. You’re at risk! Protect Yourself from Extreme Heat, vol. 14; 2011.
4.
5.
Watts N, Adger WN, Agnolucci P, Blackstock J, Byass P, Cai W, et al. Health and climate change: policy responses to protect public health. Lancet. 2015;386(10006):1861–914.PubMedCrossRef
6.
Berry P, Clarke K, Fleury MD, Parker S. Human health. In: Warren FJ, Lemmen DS, editors. Canada in a changing climate: sector perspectives on impacts and adaptation. Ottawa, ON, Canada: Government of Canada; 2014. p. 191–232.
7.
8.
Khatchikian CE, Prusinski MA, Stone M, Backenson PB, Wang I-N, Foley E, et al. Recent and rapid population growth and range expansion of the Lyme disease tick vector, Ixodes scapularis, in North America. Evolution. 2015;69(7):1678–89.PubMedCrossRefPubMedCentral
9.
McPherson M, García-García A, Cuesta-Valero FJ, Beltrami H, Hansen-Ketchum P, MacDougall D, et al. Expansion of the Lyme disease vector Ixodes Scapularis in Canada inferred from CMIP5 climate projections. Environ Health Perspect. 2017;125(5):057008.PubMedPubMedCentralCrossRef
10.
Ogden NH, Radojević M, Wu X, Duvvuri VR, Leighton PA, Wu J. Estimated effects of projected climate change on the basic reproductive number of the Lyme disease vector ixodes scapularis. Environ Health Perspect. 2014;122(6):631–8.PubMedPubMedCentralCrossRef
11.
Simon JA, Marrotte RR, Desrosiers N, Fiset J, Gaitan J, Gonzalez A, et al. Climate change and habitat fragmentation drive the occurrence of Borrelia burgdorferi, the agent of Lyme disease, at the northeastern limit of its distribution. Evol Appl. 2014;7(7):750–64.PubMedPubMedCentralCrossRef
12.
Wu X, Lu Y, Zhou S, Chen L, Xu B. Impact of climate change on human infectious diseases: empirical evidence and human adaptation. Environ Int. 2016;86:14–23.PubMedCrossRef
13.
Bouchard C, Leonard E, Koffi JK, Pelcat Y, Peregrine A, Chilton N, et al. The increasing risk of Lyme disease in Canada. Can Vet J. 2015;56(7):693.PubMedPubMedCentral
14.
15.
16.
Bethlehem J, Biffignandi S. Handbook of web surveys, vol. 567. Hoboken: Wiley; 2011.
17.
Tourangeau R, Conrad FG, Couper MP. The science of web surveys. Oxford: Oxford University Press; 2013. p. 208.
18.
Groves RM, Fowler FJ Jr, Couper MP, Lepkowski JM, Singer E, Tourangeau R. Survey methodology, 2nd edition. 2nd ed; 2009.
19.
Institut de la Statistique du Québec. L’Enquête québécoise sur l’accès des ménages à Internet 2012. 2013.
20.
Poushter J. Smartphone ownership and internet usage continues to climb in emerging economies. Pew Res Center. 2016;22:1–44.
21.
Sterrett D, Malato D, Benz J, Tompson T, English N. Assessing changes in coverage Bias of web surveys in the United States. Public Opin Q. 2017;81(S1):338–56.CrossRef
22.
Chang L, Krosnick JA. National surveys via RDD telephone interviewing versus the internet: comparing sample representativeness and response quality. Public Opin Q. 2009;73(4):641–78.CrossRef
23.
24.
Fricker S, Galesic M, Tourangeau R, Yan T. An experimental comparison of web and telephone surveys. Public Opin Q. 2005;69(3):370–92.CrossRef
25.
Gingras M-È, Belleau H. Avantages et désavantages du sondage en ligne comme méthode de collecte de données: une revue de la littérature; 2015.
26.
Sarracino F, Riillo CFA, Mikucka M. Comparability of web and telephone survey modes for the measurement of subjective well-being. Survey Res Methods. 2017;11(2):141–69.
27.
Stephenson LB, Crête J. Studying political behavior: a comparison of internet and telephone surveys. Int J Public Opin Res. 2010;23(1):24–55.CrossRef
28.
Zhang X, Kuchinke L, Woud ML, Velten J, Margraf J. Survey method matters: online/offline questionnaires and face-to-face or telephone interviews differ. Comput Hum Behav. 2017;71:172–80.CrossRef
29.
Keeter S, McGeeney K, Igielnik R, Mercer A, Mathiowetz N. From telephone to the web: The challenge of mode of interview effects in public opinion polls. Pew Research Center. 2015;May 13.
30.
Fan W, Yan Z. Factors affecting response rates of the web survey: a systematic review. Comput Hum Behav. 2010;26(2):132–9.CrossRef
31.
Bethlehem J. Peut-on établir des statistiques officielles à partir d’enquêtes en ligne reposant sur le principe de l’autosélection? Canada: Statistique Canada; 2008.
32.
Roster CA, Rogers RD, Albaum G, Klein D. A comparison of response characteristics from web and telephone surveys. Int J Mark Res. 2004;46:359–74.CrossRef
33.
Cefrio L. Équipement et branchement internet des foyers québécois. Netendances. 2015;6:1-12.
34.
Porter CE, Donthu N. Using the technology acceptance model to explain how attitudes determine internet usage: the role of perceived access barriers and demographics. J Bus Res. 2006;59(9):999–1007.CrossRef
35.
Gauvin S. Les régions sont-elles vraiment moins connectées. 2012http://​www.​cefrio.​qc.​ca/​media/​uploader/​carte_​ntendances_​v2_​28mai.​pdf; .
36.
Couper MP. The future of modes of data collection. Public Opin Q. 2011;75(5):889–908.CrossRef
37.
Breton C, Cutler F, Lachance S, Mierke-Zatwarnicki A. Telephone versus online survey modes for election studies: comparing Canadian public opinion and vote choice in the 2015 Federal Election. Can J Political Sci/Revue canadienne de science politique. 2017;50(4):1005–36.CrossRef
38.
39.
Rose E, DiClemente RJ, Wingood GM, Sales JM, Latham TP, Crosby RA, et al. The validity of teens’ and young adults’ self-reported condom use. Arch Pediatr Adolesc Med. 2009;163(1):61–4.PubMedCrossRef
40.
Grandjean BD, Nelson NM, Taylor PA. Comparing an internet panel survey to mail and phone surveys on willingness to pay for environmental quality: a national mode test; 2009. p. 14–7.
41.
Wright KB. Researching internet-based populations: advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. J Comput-Mediat Commun. 2005;10(3):00.CrossRef
42.
Cefrio L. Portrait numérique des foyers Québécois. Site internet du Cefrio; 2016. p. 21. http://​www.​cefrio.​qc.​ca/​media/​uploader/​Fascicule2016-Portraitnumrique​desfoyersqubcois​02112016.​pdf.​
43.
44.
45.
46.
47.
Kish L. Optima and proxima in linear sample designs. J Royal Stat Soc Ser A (General). 1976;139:80–95.
48.
Kish L. Multipurpose sample designs. Surv Methodol. 1988;14(1):19–32.
49.
Kish L. Survey sampling. New York: Wiley; 1965.
50.
Vázquez M, Muehlenbein C, Cartter M, Hayes EB, Ertel S, Shapiro ED. Effectiveness of personal protective measures to prevent Lyme disease. Emerg Infect Dis. 2008;14(2):210–6.PubMedPubMedCentralCrossRef
51.
Smith G, Wileyto EP, Hopkins RB, Cherry BR, Maher JP. Risk factors for Lyme disease in Chester County, Pennsylvania. Public Health Rep. 2001;116(1 suppl):146–56.PubMedPubMedCentralCrossRef
52.
Aenishaenslin C, Bouchard C, Koffi JK, Ogden NH. Exposure and preventive behaviours toward ticks and Lyme disease in Canada: results from a first national survey. Ticks Tick-Borne Dis. 2017;8(1):112–8.PubMedCrossRef
53.
Beaujean DJMA, Bults M, van Steenbergen JE, Voeten HACM. Study on public perceptions and protective behaviors regarding Lyme disease among the general public in the Netherlands: implications for prevention programs. BMC Public Health. 2013;13(1):225.PubMedPubMedCentralCrossRef
54.
Beaujean D, Van Velsen L, Van Gemert-Pijnen JEWC, Maat A, Van Steenbergen JE, Crutzen R. Using risk group profiles as a lightweight qualitative approach for intervention development: An example of prevention of tick bites and lyme disease. J Med Internet Res Protocols. 2013;2(2):e45.
55.
Bouchard C, Aenishaenslin C, Rees EE, Koffi JK, Pelcat Y, Ripoche M, et al. Integrated social-behavioral and ecological risk maps to prioritize local public health responses to Lyme disease. Environ Health Perspect. 2018;126(4):047008.PubMedPubMedCentralCrossRef
56.
57.
Valois P, Bouchard D, Aenishaenslin C, Talbot D, Bouchard C, Briand S, et al. Development and Validation of a Behavioral Index for Adaptation to Lyme Disease. Manuscript submitted for publication.
58.
Cohen. Statistical power analysis for the behavioural sciences. 2nd ed. Hilldale, NJ: Lawrence Erlbaum Associates Publishers; 1988. p. 567.
59.
Millsap E. Statistical methods for studying measurement invariance. Abingdon: Taylor and Francis; 2011.
60.
Morin A, Marsh H, Nagengast B. Exploratory structural equation modeling. In: Hancock G, Mueller R, editors. Structural equation modeling: A second course; 2013. p. 395–436.
61.
Meitinger K. Necessary but InsufficientWhy measurement invariance tests need online probing as a complementary tool. Public Opin Q. 2017;81(2):447–72.PubMedPubMedCentralCrossRef
62.
Muthén LK, Muthén BO. Mplus User’s Guide. 7th ed. Los Angeles: Muthén & Muthén; 2015.
63.
Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model Multidiscip J. 1999;6(1):1–55.CrossRef
64.
Kline R. Principles and practice of structural equation modeling. 3rd ed. New York: Guildford Publications; 2011. 427.
65.
Deville J-C, Särndal C-E. Calibration estimators in survey sampling. J Am Stat Assoc. 1992;87(418):376–82.CrossRef
66.
The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcomes Rates for Surveys. AAPOR. 9th ed; 2016. p. 81. Report No.
67.
Dutwin D, Buskirk TD. Apples to oranges or gala versus golden delicious? Comparing data quality of nonprobability internet samples to low response rate probability samples. Public Opin Q. 2017;81(S1):213–39.CrossRef
68.
MacInnis B, Krosnick JA, S Ho A, Cho M-J. The Accuracy of Measurements with Probability and Nonprobability Survey SamplesReplication and Extension. Public Opin Q. 2018;82:707-44.
69.
Yeager DS, Krosnick JA, Chang L, Javitz HS, Levendusky MS, Simpser A, et al. Comparing the accuracy of RDD telephone surveys and internet surveys conducted with probability and non-probability samples. Public Opin Q. 2011;75(4):709–47.CrossRef
70.
Yan T, Curtin R, Jans M. Trends in income nonresponse over two decades. J Off Stat. 2010;26(1):145.
71.
Decieux JPP, Mergener A, Sischka P, Neufang K. Implementation of the forced answering option within online surveys: do higher item response rates come at the expense of participation and answer quality? Psihologija. 2015;48(4):311–26.CrossRef
72.
Sischka P, Decieux JPP, Mergener A, Neufang K. Forced answering in online surveys: do higher item response rates come at the expense of participation and answer quality? 2016.
73.
Ajzen I. Attitudes, personality, and behavior. UK: McGraw-Hill Education; 2005.
74.
Sala E, Lillini R. Undercoverage Bias in telephone surveys in Europe: the Italian case. Int J Public Opin Res. 2015;29(1):133–56.
75.
Tourangeau R. Presidential address: paradoxes of nonresponse. Public Opin Q. 2017;81(3):803–14.CrossRef
76.
Williams D, Brick JM. Trends in U.S. face-to-face household survey nonresponse and level of effort. J Surv Stat Methodol. 2018;6(2):186–211.CrossRef
77.
78.
Bethlehem J, Cobben F, Schouten B. Des indicateurs de la représentativité des réponses aux enquêtes. Canada: Statistique Canada; 2008. p. 1–10.
79.
Braunsberger K, Wybenga H, Gates R. A comparison of reliability between telephone and web-based surveys. J Bus Res. 2007;60(7):758–64.CrossRef
Metadata
Title
Telephone versus web panel National Survey for monitoring adoption of preventive behaviors to climate change in populations: a case study of Lyme disease in Québec, Canada
Authors
Grâce Ngambo Domche
Pierre Valois
Magalie Canuel
Denis Talbot
Maxime Tessier
Cécile Aenishaenslin
Catherine Bouchard
Sandie Briand
Publication date
01-12-2020
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2020
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-020-00958-4

Other articles of this Issue 1/2020

BMC Medical Research Methodology 1/2020 Go to the issue

Research article

Predictive accuracy of a hierarchical logistic model of cumulative SARS-CoV-2 case growth until May 2020

Research article

A mixed methods case study investigating how randomised controlled trials (RCTs) are reported, understood and interpreted in practice

Research article

Multiple imputation with missing indicators as proxies for unmeasured variables: simulation study

Research article

Performance of unanchored matching-adjusted indirect comparison (MAIC) for the evidence synthesis of single-arm trials with time-to-event outcomes

Research article

Predictive approaches to heterogeneous treatment effects: a scoping review

Technical advance

Assessing risk of bias: a proposal for a unified framework for observational studies and randomized trials