Skip to main content
Top
Published in: The European Journal of Health Economics 6/2012

01-12-2012 | Original Paper

The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey

Authors: Aleksandra Gajic, David Cameron, Jeremiah Hurley

Published in: The European Journal of Health Economics | Issue 6/2012

Login to get access

Abstract

We present the results of a randomized experiment to test the effectiveness and cost-effectiveness of response incentives for a stated-preference survey of a general community population. The survey was administered using a mixed-mode approach, in which community members were invited to participate using a traditional mailed letter using contact information for a representative sample of the community; but individuals completed the survey via the web, which exploited the advantages of electronic capture. Individuals were randomized to four incentive groups: (a) no incentive, (b) prepaid cash incentive ($2), (c) a low lottery (10 prizes of $25) and (d) a high lottery (2 prizes of $250). Letters of invitation were mailed to 3,000 individuals. In total, 405 individuals (14.4%) contacted the website and 277 (9.8%) provided complete responses. The prepaid cash incentive generated the highest contact and response rates (23.3 and 17.3%, respectively), and no incentive generated the lowest (9.1 and 5.7%, respectively). The high lottery, however, was the most cost-effective incentive for obtaining completed surveys: compared with no incentive, the incremental cost-effectiveness ratio (ICER) per completed survey for high lottery was $13.89; for prepaid cash, the ICER was $18.29. This finding suggests that the preferred response incentive for community-based, stated-preference surveys is a lottery with a small number of large prizes.
Footnotes
1
Response rates are less well documented for health economic surveys, but the experience of those who regularly conduct stated preference surveys indicates that response rates have been falling in recent years [10].
 
2
This includes individuals who may not have access to the Internet. Because we have no way of identifying these individuals, we must assume all those who received the letter were eligible to respond.
 
Literature
1.
go back to reference Ryan, M., Gerard, K., Amaya–Amaya, M.: Using Discrete Choice Experiments to Value Health and Health Care. Springer, Dordrecht (2008)CrossRef Ryan, M., Gerard, K., Amaya–Amaya, M.: Using Discrete Choice Experiments to Value Health and Health Care. Springer, Dordrecht (2008)CrossRef
2.
go back to reference Hurley, J., Buckley, N., Cuff, K., Giacomini, M., Cameron, D.: Judgments regarding the fair division of goods: the impact of verbal versus quantitative descriptions of alternative principles. Social Choice and Welfare Forthcoming (2010) Hurley, J., Buckley, N., Cuff, K., Giacomini, M., Cameron, D.: Judgments regarding the fair division of goods: the impact of verbal versus quantitative descriptions of alternative principles. Social Choice and Welfare Forthcoming (2010)
3.
go back to reference Nord, E., Richardson, J., Street, A., Kuhse, H., Singer, P.: Maximizing health benefits vs. Egalitarianism: an Australian survey of health issues. Soc. Sci. Med. 41(10), 1429–1437 (1995)PubMedCrossRef Nord, E., Richardson, J., Street, A., Kuhse, H., Singer, P.: Maximizing health benefits vs. Egalitarianism: an Australian survey of health issues. Soc. Sci. Med. 41(10), 1429–1437 (1995)PubMedCrossRef
4.
go back to reference Dolan, P., Tsuchiya, A.: Health priorities and public preferences: the relative importance of past health experience and future health prospects. J. Health Econ. 24, 703–714 (2005)PubMedCrossRef Dolan, P., Tsuchiya, A.: Health priorities and public preferences: the relative importance of past health experience and future health prospects. J. Health Econ. 24, 703–714 (2005)PubMedCrossRef
5.
go back to reference Sheehan, K.: E-mail survey response rates: A review. J. Comput. Mediat. Commun. 6(2). (2001) Sheehan, K.: E-mail survey response rates: A review. J. Comput. Mediat. Commun. 6(2). (2001)
6.
go back to reference Warriner, K., Goyder, J., Gjersten, H., honhner, P., McSpurren, K.: Charities, no; lotteries, no; cash, yes. Main effects and interactions in a Canadian incentives experiment. Public Opin Q 60, 542–562 (1996)CrossRef Warriner, K., Goyder, J., Gjersten, H., honhner, P., McSpurren, K.: Charities, no; lotteries, no; cash, yes. Main effects and interactions in a Canadian incentives experiment. Public Opin Q 60, 542–562 (1996)CrossRef
7.
go back to reference Bickart, B., Schmittlein, D.: The distribution of survey contact and participation in the United States: constructing a survey-based estimate. J. Mark. Res. 36(2), 286–294 (1999)CrossRef Bickart, B., Schmittlein, D.: The distribution of survey contact and participation in the United States: constructing a survey-based estimate. J. Mark. Res. 36(2), 286–294 (1999)CrossRef
8.
go back to reference Kulka, R., Eyerman, J., McNeeley, M.: The use of monetary incentives in federal surveys on substance use and abuse. J. Econ. Soc. Meas. 30(2–3), 233–249 (2005) Kulka, R., Eyerman, J., McNeeley, M.: The use of monetary incentives in federal surveys on substance use and abuse. J. Econ. Soc. Meas. 30(2–3), 233–249 (2005)
9.
go back to reference Dixon, J., Tucker, C.: Survey nonresponse. In: Marsden, P., Wright, J. (eds.) Handbook of Survey Research. Emerald Publishing Group, Bingley, UK (2010) Dixon, J., Tucker, C.: Survey nonresponse. In: Marsden, P., Wright, J. (eds.) Handbook of Survey Research. Emerald Publishing Group, Bingley, UK (2010)
10.
go back to reference Ryan, M.: Personal E-mail Communication, August 17. (2010) Ryan, M.: Personal E-mail Communication, August 17. (2010)
11.
go back to reference Bosnjak, M., Tuten, T.: Prepaid and promised incentives in web surveys. Soc. Sci. Comp. Rev. 21(2), 208–217 (2003)CrossRef Bosnjak, M., Tuten, T.: Prepaid and promised incentives in web surveys. Soc. Sci. Comp. Rev. 21(2), 208–217 (2003)CrossRef
12.
go back to reference Umbach, P.D.: Web-based surveys: best practices. New Dir. Inst. Res. 121, 23–38 (2004) Umbach, P.D.: Web-based surveys: best practices. New Dir. Inst. Res. 121, 23–38 (2004)
13.
go back to reference Heerwegh, D.: An investigation of the effect of lotteries on web survey response rates. Field Methods 18(2), 205–220 (2006)CrossRef Heerwegh, D.: An investigation of the effect of lotteries on web survey response rates. Field Methods 18(2), 205–220 (2006)CrossRef
14.
go back to reference Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., Kwan, I.: Increasing response rates to postal questionnaires: a systematic review. BMJ 324, 1–9 (2002) Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., Kwan, I.: Increasing response rates to postal questionnaires: a systematic review. BMJ 324, 1–9 (2002)
15.
go back to reference Armstrong, J.S.: Monetary incentives in mailed surveys. Public Opin. Q. 39, 111–116 (1975)CrossRef Armstrong, J.S.: Monetary incentives in mailed surveys. Public Opin. Q. 39, 111–116 (1975)CrossRef
16.
go back to reference Linsky, A.S.: Stimulating responses to mailed questionnaires: a review. Public Opin. Q. 39, 82–101 (1975)CrossRef Linsky, A.S.: Stimulating responses to mailed questionnaires: a review. Public Opin. Q. 39, 82–101 (1975)CrossRef
17.
go back to reference Church, A.H.: Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opin. Q. 57, 62–79 (1993)CrossRef Church, A.H.: Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opin. Q. 57, 62–79 (1993)CrossRef
18.
go back to reference Leung, G.M., Ho, L.M., Chan, M.F., Johnston, J.M., Wong, F.K.: The effects of cash and lottery incentives on mailed surveys to physicians. A randomized trial. J. Clin. Epidemiol. 55, 801–807 (2002)PubMedCrossRef Leung, G.M., Ho, L.M., Chan, M.F., Johnston, J.M., Wong, F.K.: The effects of cash and lottery incentives on mailed surveys to physicians. A randomized trial. J. Clin. Epidemiol. 55, 801–807 (2002)PubMedCrossRef
19.
go back to reference Porter, S.R., Whitcomb, M.E.: The impact of lottery incentives on student survey response rates. Res. High. Educ. 44(4), 389–407 (2003)CrossRef Porter, S.R., Whitcomb, M.E.: The impact of lottery incentives on student survey response rates. Res. High. Educ. 44(4), 389–407 (2003)CrossRef
20.
go back to reference Göritz, A.S.: Incentives in web-based studies: methodological issues and review. Int. J. Internet Sci. 1, 58–70 (2006) Göritz, A.S.: Incentives in web-based studies: methodological issues and review. Int. J. Internet Sci. 1, 58–70 (2006)
21.
go back to reference Frickl, A., Bachtiger, M.T., Reips, U.-D.: Financial incentives, personal information and drop-out rate in online studies. In: Reips, U.-D., Bosnjak, M. (eds.) Dimensions of Internet Science, pp. 209–219. Pabst Science Publishers, Berlin (2001) Frickl, A., Bachtiger, M.T., Reips, U.-D.: Financial incentives, personal information and drop-out rate in online studies. In: Reips, U.-D., Bosnjak, M. (eds.) Dimensions of Internet Science, pp. 209–219. Pabst Science Publishers, Berlin (2001)
22.
go back to reference Duetskens, E., Ruyter, K.D., Wetzels, M., Oosterveld, P.: Response rate and response quality of internet-based surveys: an experimental study. Mark. Lett. 15(1), 21–36 (2004)CrossRef Duetskens, E., Ruyter, K.D., Wetzels, M., Oosterveld, P.: Response rate and response quality of internet-based surveys: an experimental study. Mark. Lett. 15(1), 21–36 (2004)CrossRef
23.
go back to reference Göritz, A.S.: The impact of material incentives on response quality, sample composition, survey income, and cost in online access panels. Int. J. Mark. Res. 46, 327–345 (2004) Göritz, A.S.: The impact of material incentives on response quality, sample composition, survey income, and cost in online access panels. Int. J. Mark. Res. 46, 327–345 (2004)
24.
go back to reference Su, J., Shao, S., Fang, J.: Effects of incentives on web-based surveys. Tsinghua Sci. Technol. 13(3), 344–347 (2008)CrossRef Su, J., Shao, S., Fang, J.: Effects of incentives on web-based surveys. Tsinghua Sci. Technol. 13(3), 344–347 (2008)CrossRef
25.
go back to reference Birnholtz, J.P., Horn, D.B., Finholt, T.A., Bae, S.J.: The effects of cash, electronic and paper gift certificates for a web-based survey of technologically sophisticated respondents. Soc. Sci. Comp. Rev. 22, 355–362 (2004)CrossRef Birnholtz, J.P., Horn, D.B., Finholt, T.A., Bae, S.J.: The effects of cash, electronic and paper gift certificates for a web-based survey of technologically sophisticated respondents. Soc. Sci. Comp. Rev. 22, 355–362 (2004)CrossRef
26.
go back to reference Smith, R.: The discrete-choice willingness-to-pay question format in health economics: should we adopt environmental guidelines. Med. Decis. Mak. 20(2), 194–204 (2000)CrossRef Smith, R.: The discrete-choice willingness-to-pay question format in health economics: should we adopt environmental guidelines. Med. Decis. Mak. 20(2), 194–204 (2000)CrossRef
27.
go back to reference Messick, D.M., McClintock, C.G.: Motivational bases of choice n experimental games. J. Exp. Soc. Psychol. 4, 1–25 (1968)CrossRef Messick, D.M., McClintock, C.G.: Motivational bases of choice n experimental games. J. Exp. Soc. Psychol. 4, 1–25 (1968)CrossRef
28.
go back to reference Hurley, J., Mentzakis, E.: The Existence and Magnitude of Health Care Externalities: Evidence from a Choice Experiment. Hamilton, ON: McMaster University Centre for Health Economics and Policy Analysis Working Paper 11–01. (2011) Hurley, J., Mentzakis, E.: The Existence and Magnitude of Health Care Externalities: Evidence from a Choice Experiment. Hamilton, ON: McMaster University Centre for Health Economics and Policy Analysis Working Paper 11–01. (2011)
29.
go back to reference Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.: Mail and Internet Survey: The Tailored Design Method. Wiley, New York (2007) Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.: Mail and Internet Survey: The Tailored Design Method. Wiley, New York (2007)
30.
go back to reference Offerman, T., Sonnemans, J., Schram, A.: Value orientations, expectations and voluntary contributions in public goods. Econ. J. 106(437), 817–845 (1996)CrossRef Offerman, T., Sonnemans, J., Schram, A.: Value orientations, expectations and voluntary contributions in public goods. Econ. J. 106(437), 817–845 (1996)CrossRef
31.
go back to reference Buckley, N., Chan, K.S., Chowhan, J., Mestelman, S., Shehata, M.: Value orientations, income and displacement effects, and voluntary contributions. Exp. Econ. 4(2), 183–195 (2001) Buckley, N., Chan, K.S., Chowhan, J., Mestelman, S., Shehata, M.: Value orientations, income and displacement effects, and voluntary contributions. Exp. Econ. 4(2), 183–195 (2001)
32.
go back to reference Dillman, D.A.: Response rate and measurement differences in mixed mode surveys using mail, telephone, interactive voice response and the internet. Soc. Sci. Res. 38(1), 1–18 (2009)CrossRef Dillman, D.A.: Response rate and measurement differences in mixed mode surveys using mail, telephone, interactive voice response and the internet. Soc. Sci. Res. 38(1), 1–18 (2009)CrossRef
33.
go back to reference Manfreda, L., Bosnjak, M., Haas, J., Vehovar, V.: Web surveys versus other survey modes: a meta-analysis comparing response rates. Int. J. Mark. Res. 50, 79–104 (2008) Manfreda, L., Bosnjak, M., Haas, J., Vehovar, V.: Web surveys versus other survey modes: a meta-analysis comparing response rates. Int. J. Mark. Res. 50, 79–104 (2008)
Metadata
Title
The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey
Authors
Aleksandra Gajic
David Cameron
Jeremiah Hurley
Publication date
01-12-2012
Publisher
Springer-Verlag
Published in
The European Journal of Health Economics / Issue 6/2012
Print ISSN: 1618-7598
Electronic ISSN: 1618-7601
DOI
https://doi.org/10.1007/s10198-011-0332-0

Other articles of this Issue 6/2012

The European Journal of Health Economics 6/2012 Go to the issue