Abstract
This experiment investigated the effects of survey administration mode (Web vs. paper and pencil) and survey length (short or long) on the responses of a large sample of primary and secondary school professionals in the U.S. The 812 participants in this study were part of an initial random sample of 1,000 individuals representing 5.81% of the membership of a national professional organization. The participants were randomly assigned to each of the four treatment conditions. Results indicated (1) substantially lower response rates for Web surveys than for same-length paper surveys; (2) a higher response rate for short surveys than for long surveys with paper, but not Web, questionnaires; and (3) a younger age for Web respondents, as compared with their paper counterparts. In light of prior research, we suggest that paper-and-pencil methods be used for surveying professionals in primary and secondary school settings.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bachmann, D. P., Elfrink, J., & Vazzana, G. (2000). E-mail and snail mail face off in rematch. Marketing Research, 11, 10–15.
Birnbaum, M. H. (1999). Testing critical properties of decision making on the Internet. Psychological Science, 10, 399–407. doi:10.1111/1467-9280.00176
Cobanoglu, C., Warde, B., & Moreo, P. J. (2001). A comparison of mail, fax, and Web-based survey methods. International Journal of Market Research, 43, 441–455.
Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in Web- or Internet-based surveys. Educational & Psychological Measurement, 60, 821–836. doi:10.1177/00131640021970934
Couper, M. P. (2005). Technology trends in survey data collection. Social Science Computer Review, 23, 486–501. doi:10.1177/ 0894439305278972
Curtis, M. J., Hunley, S. A., Walker, K. J., & Baker, A. C. (1999). Demographic characteristics and professional practices in school psychology. School Psychology Review, 28, 104–116.
Curtis, M. J., Lopez, A. D., Batsche, G. M., & Smith, J. C. (2006, March). School psychology 2005: A national perspective. Paper presented at the meeting of the National Association of School Psychologists, Anaheim, CA.
Dillman, D. (2000). Main and Internet surveys: The tailored design method (2nd ed.). New York: Wiley.
Diment, K., & Garrett-Jones, S. (2007). How demographic characteristics affect mode preference in a postal/Web mixed-mode survey of Australian researchers. Social Science Computer Review, 25, 410–417. Retrieved May 4, 2009, from http://ro.uow.edu.au/commpapers/304/. doi:10.1177/0894439306295393
Fagan, T. K., Gorin, S., & Tharinger, D. (2000). The National Association of School Psychologists and the Division of School Psychology-APA: Now and beyond. School Psychology Review, 29, 525–535.
Feldt, L. S., & Ankenmann, R. D. (1999). Determining sample size for a test of the equality of alpha coefficients when the number of parttests is small. Psychological Methods, 4, 366–377. doi:10.1037/1082-989X.4.4.366
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust Web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59, 93–104. doi:10.1037/0003-066X.59.2.93
Heberlein, T. A., & Baumgartner, R. (1978). Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. American Sociological Review, 43, 447–462.
Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York: Wiley.
Individuals With Disabilities Education Improvement Act of 2004. 20 U.S.C. 1400.
Kaplowitz, M. D., Hadlock, T. D., & Levine, R. (2004). A comparison of Web and mail survey response rates. Public Opinion Quarterly, 68, 94–101. doi:10.1093/poq/nfh006
LaRose, R., Mastro, D., & Eastin, M. S. (2001). Understanding Internet usage: A social-cognitive approach to uses and gratifications. Social Science Computer Review, 19, 395–413.
Marcus, B., Bosnjak, M., Lindner, S., Pilischenko, S., & Schutz, A. (2007). Compensating for low topic interest and long surveys: A field experiment on nonresponse in Web surveys. Social Science Computer Review, 25, 372–383. doi:10.1177/0894439307297606
McCabe, S. E., Couper, M. P., Cranford, J. A., & Boyd, C. J. (2006). Comparison of Web and mail surveys for studying secondary consequences associated with substance use: Evidence for minimal mode effects. Addictive Behaviors, 31, 162–168. doi:10.1016/ j.addbeh.2005.04.018
McDonald, H., & Adam, S. (2003). A comparison of online and postal data collection methods in marketing research. Marketing Intelligence & Planning, 21, 85–95. doi:10.1108/02634500310465399
Mertler, C. A. (2003). Patterns of response and nonresponse from teachers to traditional and Web surveys. Practical Assessment, Research & Evaluation, 8, Art. 22. Retrieved January 22, 2009, from http://PAREonline.net/getvn.asp?v=8&n=22.
Microsoft Corporation (2003). Microsoft Office FrontPage (Version 2003) [Computer software]. Redmond, WA: Microsoft, Inc.
Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment & Evaluation in Higher Education, 33, 301–314. doi:10.1080/02602930701293231
Pasveer, K. A., & Ellard, J. H. (1998). The making of a personality inventory: Help from the WWW. Behavior Research Methods, Instruments, & Computers, 30, 309–313.
Reips, U.-D. (2002). Standards for Internet-based experimenting. Experimental Psychology, 49, 243–256. doi:10.1026//1618-3169.49.4.243
Reips, U.-D., & Lengler, R. (2005). The Web Experiment List: A Web service for the recruitment of participants and archiving of Internet-based experiments. Behavior Research Methods, 37, 287–292.
Sheehan, K. B. (2001). E-mail survey response rates: A review. Journal of Computer-Mediated Communication, 6(2). Retrieved April 16, 2009, from www3.interscience.wiley.com.argo.library.okstate.edu/ cgi-bin/fulltext/120837811/HTMLSTARTW.
Sheehan, K. B., & McMillan, S. J. (1999). Response variation in email surveys: An exploration. Journal of Advertising Research, 39, 45–54.
Solomon, D. J. (2001). Conducting Web-based surveys. Practical Assessment, Research & Evaluation, 7, Art. 19. Retrieved January 22, 2009, from http://PAREonline.net/getvn.asp?v=7&n=19.
Stanton, J. M., & Rogelberg, S. G. (2001). Using Internet/intranet Web pages to collect organizational research data. Organizational Research Methods, 4, 200–217. doi:10.1177/109442810143002
Weible, R., & Wallace, J. (1998). Cyber research: The impact of the Internet on data collection. Marketing Research, 10, 19–25.
Yetter, G., & Doll, B. (2007). The impact of logistical resources on pre-referral team acceptability. School Psychology Quarterly, 22, 340–357. doi:10.1037/1045-3830.22.3.340
Author information
Authors and Affiliations
Corresponding author
Additional information
Note—Accepted by the previous editorial team, when John H. Krantz was Editor.
Rights and permissions
About this article
Cite this article
Yetter, G., Capaccioli, K. Differences in responses to Web and paper surveys among school professionals. Behavior Research Methods 42, 266–272 (2010). https://doi.org/10.3758/BRM.42.1.266
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BRM.42.1.266