Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Research article

An experimental comparison of web-push vs. paper-only survey procedures for conducting an in-depth health survey of military spouses

Authors: Hope Seib McMaster, Cynthia A. LeardMann, Steven Speigle, Don A. Dillman, for the Millennium Cohort Family Study Team

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

Previous research has found that a “web-push” approach to data collection, which involves contacting people by mail to request an Internet survey response while withholding a paper response option until later in the contact process, consistently achieves lower response rates than a “paper-only” approach, whereby all respondents are contacted and requested to respond by mail.

Method

An experiment was designed, as part of the Millennium Cohort Family Study, to compare response rates, sample representativeness, and cost between a web-push and a paper-only approach; each approach comprised 3 stages of mail contacts. The invited sample (n = 4,935) consisted of spouses married to U.S. Service members, who had been serving in the military between 2 and 5 years as of October, 2011.

Results

The web-push methodology produced a significantly higher response rate, 32.8% compared to 27.8%. Each of the 3 stages of postal contact significantly contributed to response for both treatments with 87.1% of the web-push responses received over the Internet. The per-respondent cost of the paper-only treatment was almost 40% higher than the web-push treatment group. Analyses revealed no meaningfully significant differences between treatment groups in representation.

Conclusion

These results provide evidence that a web-push methodology is more effective and less expensive than a paper-only approach among young military spouses, perhaps due to their heavy reliance on the internet, and we suggest that this approach may be more effective with the general population as they become more uniformly internet savvy.
Literature
1.
go back to reference Dillman DA, Smyth JD, Christian LM. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th ed. New Jersey: John Wiley and Sons; 2014. Dillman DA, Smyth JD, Christian LM. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th ed. New Jersey: John Wiley and Sons; 2014.
2.
go back to reference Finamore J, Dillman DA. How mode sequence affects response by internet, mail, and telephone in the National Survey of College Graduates. In: Paper presented at the European Survey Research Association Biennial Conference. Ljubljana, Slovenia. 2013. Finamore J, Dillman DA. How mode sequence affects response by internet, mail, and telephone in the National Survey of College Graduates. In: Paper presented at the European Survey Research Association Biennial Conference. Ljubljana, Slovenia. 2013.
3.
go back to reference Smyth JD, Dillman DA, Christian LM, O’Neill A. Using the Internet to survey small towns and communities: Limitations and possibilities in the early 21st century. Am Behav Sci. 2010;53:1423–48.CrossRef Smyth JD, Dillman DA, Christian LM, O’Neill A. Using the Internet to survey small towns and communities: Limitations and possibilities in the early 21st century. Am Behav Sci. 2010;53:1423–48.CrossRef
4.
go back to reference Gentry R. Offering respondents a choice of survey mode. In: Paper presented at CMOR Respondent Cooperation Workshop: March 2008; Las Vegas,NV. 2008. Gentry R. Offering respondents a choice of survey mode. In: Paper presented at CMOR Respondent Cooperation Workshop: March 2008; Las Vegas,NV. 2008.
5.
go back to reference Medway R, Fulton J. When more gets you less: A meta-analysis of the effect of concurrent web options on mail survey response rates. Public Opin Q. 2012;76(4):733–46.CrossRef Medway R, Fulton J. When more gets you less: A meta-analysis of the effect of concurrent web options on mail survey response rates. Public Opin Q. 2012;76(4):733–46.CrossRef
6.
go back to reference Schwarz N, Donsbach W, Traugott MW. The psychology of survey response. In: The Sage handbook of public opinion research. Newbury Park, CA: Sage; 2008. p. 374–87.CrossRef Schwarz N, Donsbach W, Traugott MW. The psychology of survey response. In: The Sage handbook of public opinion research. Newbury Park, CA: Sage; 2008. p. 374–87.CrossRef
7.
go back to reference Jacobson IG, Ryan MA, Hooper TI, et al. Alcohol use and alcohol-related problems before and after military combat deployment. J Am Med Assoc. 2008;300(6):663–75.CrossRef Jacobson IG, Ryan MA, Hooper TI, et al. Alcohol use and alcohol-related problems before and after military combat deployment. J Am Med Assoc. 2008;300(6):663–75.CrossRef
8.
go back to reference Ryan MA, Smith TC, Smith B, Amoroso P, Boyko EJ, Gray GC, Gackstetter GD, Riddle JR, Wells TS, Gumbs G, et al. Millennium Cohort: enrollment begins a 21-year contribution to understanding the impact of military service. J Clin Epidemiol. 2007;60(2):181–91.CrossRefPubMed Ryan MA, Smith TC, Smith B, Amoroso P, Boyko EJ, Gray GC, Gackstetter GD, Riddle JR, Wells TS, Gumbs G, et al. Millennium Cohort: enrollment begins a 21-year contribution to understanding the impact of military service. J Clin Epidemiol. 2007;60(2):181–91.CrossRefPubMed
9.
go back to reference Smith TC, Jacobson IG, Hooper TI, LeardMann CA, Boyko EJ, Smith B, Gackstetter GD, Wells TS, Amoroso PJ, Gray GC, et al. Health impact of US military service in a large population-based military cohort: findings of the Millennium Cohort Study 2001–2008. BMC Public Health. 2011;11:69.CrossRefPubMedPubMedCentral Smith TC, Jacobson IG, Hooper TI, LeardMann CA, Boyko EJ, Smith B, Gackstetter GD, Wells TS, Amoroso PJ, Gray GC, et al. Health impact of US military service in a large population-based military cohort: findings of the Millennium Cohort Study 2001–2008. BMC Public Health. 2011;11:69.CrossRefPubMedPubMedCentral
10.
go back to reference Smith TC, Ryan MA, Wingard DL, Slymen DJ, Sallis JF, Kritz-Silverstein D. New onset and persistent symptoms of posttraumatic stress disorder self-reported after deployment and combat exposures: prospective population based US military cohort study. Br Med J. 2008;336:366–71.CrossRef Smith TC, Ryan MA, Wingard DL, Slymen DJ, Sallis JF, Kritz-Silverstein D. New onset and persistent symptoms of posttraumatic stress disorder self-reported after deployment and combat exposures: prospective population based US military cohort study. Br Med J. 2008;336:366–71.CrossRef
11.
go back to reference Wang L, Seelig A, MacDermid S, McMaster H, Alcaraz JE, Crum-Cianflone NF: Associations of military divorce with mental, behavioral, and physical health outcomes. BMC Psychiatry. 2015;15(128). doi:10.1186/s12888-015-0517-7. Wang L, Seelig A, MacDermid S, McMaster H, Alcaraz JE, Crum-Cianflone NF: Associations of military divorce with mental, behavioral, and physical health outcomes. BMC Psychiatry. 2015;15(128). ​doi:10.​1186/​s12888-015-0517-7.
12.
go back to reference Dillman DA, Smyth JD, Christian LM: Internet, mail and mixed-mode surveys; The tailored design method, 3rd edition. New Jersey: Wiley; 2008. Dillman DA, Smyth JD, Christian LM: Internet, mail and mixed-mode surveys; The tailored design method, 3rd edition. New Jersey: Wiley; 2008.
13.
go back to reference Corry NH, Williams CS, Battaglia M, McMaster HS, Stander VA. Assessing and adjusting for non-response in the Millennium Cohort Family Study. BMC Medical Research Methodology. 2017;17:16. doi:10.1186/s12874-017-0294-8. Corry NH, Williams CS, Battaglia M, McMaster HS, Stander VA. Assessing and adjusting for non-response in the Millennium Cohort Family Study. BMC Medical Research Methodology. 2017;17:16. doi:10.​1186/​s12874-017-0294-8.
15.
go back to reference Pew Research Center. Summer tracking survey August 7 - September 6. In: Pew Internet and American Life Project. 2012. Pew Research Center. Summer tracking survey August 7 - September 6. In: Pew Internet and American Life Project. 2012.
16.
go back to reference Greentree V, Bradbard D, Dagher L, Levingston K, Elquist Lore C, Taylor JA. 2013 Military Families Lifestyle Report. 2013. Blue Star Families. Greentree V, Bradbard D, Dagher L, Levingston K, Elquist Lore C, Taylor JA. 2013 Military Families Lifestyle Report. 2013. Blue Star Families.
17.
go back to reference Millar MM, Dillman DA. Improving Response to Web and Mixed Mode Surveys. Public Opin Q. 2011;75(2):249–69.CrossRef Millar MM, Dillman DA. Improving Response to Web and Mixed Mode Surveys. Public Opin Q. 2011;75(2):249–69.CrossRef
18.
go back to reference Millar MM: Determining whether research is intediscipinarly: An analysis of new indicators (Technical Report No. 13-049). Pullman Washington State University; 2013. Millar MM: Determining whether research is intediscipinarly: An analysis of new indicators (Technical Report No. 13-049). Pullman Washington State University; 2013.
19.
go back to reference National research council: Reducing response burden in the American community survey: Proceedings of a workshop. The National Academies Press. 2016. National research council: Reducing response burden in the American community survey: Proceedings of a workshop. The National Academies Press. 2016.
Metadata
Title
An experimental comparison of web-push vs. paper-only survey procedures for conducting an in-depth health survey of military spouses
Authors
Hope Seib McMaster
Cynthia A. LeardMann
Steven Speigle
Don A. Dillman
for the Millennium Cohort Family Study Team
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-017-0337-1

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue