Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2011

Open Access 01-12-2011 | Research article

A comparison of small monetary incentives to convert survey non-respondents: a randomized control trial

Authors: Joan M Griffin, Alisha Baines Simon, Erin Hulbert, John Stevenson, Joseph P Grill, Siamak Noorbaloochi, Melissa R Partin

Published in: BMC Medical Research Methodology | Issue 1/2011

Login to get access

Abstract

Background

Maximizing response rates is critically important in order to provide the most generalizable and unbiased research results. High response rates reduce the chance of respondents being systematically different from non-respondents, and thus, reduce the risk of results not truly reflecting the study population. Monetary incentives are often used to improve response rates, but little is known about whether larger incentives improve response rates in those who previously have been unenthusiastic about participating in research. In this study we compared the response rates and cost-effectiveness of a $5 versus $2 monetary incentive accompanying a short survey mailed to patients who did not respond or refused to participate in research study with a face-to-face survey.

Methods

1,328 non-responders were randomly assigned to receive $5 or $2 and a short, 10-question survey by mail. Reminder postcards were sent to everyone; those not returning the survey were sent a second survey without incentive. Overall response rates, response rates by incentive condition, and odds of responding to the larger incentive were calculated. Total costs (materials, postage, and labor) and incremental cost-effectiveness ratios were also calculated and compared by incentive condition.

Results

After the first mailing, the response rate within the $5 group was significantly higher (57.8% vs. 47.7%, p < .001); after the second mailing, the difference narrowed by 80%, resulting in a non-significant difference in cumulative rates between the $5 and $2 groups (67.3% vs. 65.4%, respectively, p = .47). Regardless of incentive or number of contacts, respondents were significantly more likely to be male, white, married, and 50-75 years old. Total costs were higher with the larger versus smaller incentive ($13.77 versus $9.95 per completed survey).

Conclusions

A $5 incentive provides a significantly higher response rate than a $2 incentive if only one survey mailing is used but not if two survey mailings are used.
Appendix
Available only for authorised users
Literature
1.
go back to reference Partin MR, Malone M, Winnett M, Slater J, Bar-Cohen A, Caplan L: The impact of survey nonresponse bias on conclusions drawn from a mammography intervention trial. J Clin Epidemiol. 2003, 56: 867-873. 10.1016/S0895-4356(03)00061-1.CrossRefPubMed Partin MR, Malone M, Winnett M, Slater J, Bar-Cohen A, Caplan L: The impact of survey nonresponse bias on conclusions drawn from a mammography intervention trial. J Clin Epidemiol. 2003, 56: 867-873. 10.1016/S0895-4356(03)00061-1.CrossRefPubMed
2.
go back to reference Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al: Increasing response rates to postal questionnaires: systematic review. BMJ (Clinical research ed). 2002, 324: 1183-10.1136/bmj.324.7347.1183.CrossRefPubMedCentral Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al: Increasing response rates to postal questionnaires: systematic review. BMJ (Clinical research ed). 2002, 324: 1183-10.1136/bmj.324.7347.1183.CrossRefPubMedCentral
3.
go back to reference Edwards P, Cooper R, Roberts I, Frost C: Meta-analysis of randomised trials of monetary incentives and response to mailed questionnaires. J Epidemiol Community Health. 2005, 59: 987-999. 10.1136/jech.2005.034397.CrossRefPubMedPubMedCentral Edwards P, Cooper R, Roberts I, Frost C: Meta-analysis of randomised trials of monetary incentives and response to mailed questionnaires. J Epidemiol Community Health. 2005, 59: 987-999. 10.1136/jech.2005.034397.CrossRefPubMedPubMedCentral
4.
go back to reference Church AH: Incentives in Mailed Surveys: A Meta-Analysis. Public Opin Q. 1993, 57: 62-79. 10.1086/269355.CrossRef Church AH: Incentives in Mailed Surveys: A Meta-Analysis. Public Opin Q. 1993, 57: 62-79. 10.1086/269355.CrossRef
5.
go back to reference Singer E, Bossarte RM: Incentives for survey participation when are they "coercive"?. American Journal of Preventive Medicine. 2006, 31 (5): 411-8. 10.1016/j.amepre.2006.07.013.CrossRefPubMed Singer E, Bossarte RM: Incentives for survey participation when are they "coercive"?. American Journal of Preventive Medicine. 2006, 31 (5): 411-8. 10.1016/j.amepre.2006.07.013.CrossRefPubMed
6.
go back to reference Wertheimer A, Miller FG: Payment for research participation: a coercive offer?. Journal of Medical Ethics. 2008, 34: 389-392. 10.1136/jme.2007.021857.CrossRefPubMed Wertheimer A, Miller FG: Payment for research participation: a coercive offer?. Journal of Medical Ethics. 2008, 34: 389-392. 10.1136/jme.2007.021857.CrossRefPubMed
7.
8.
go back to reference Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al: Methods to increase response rates to postal questionnaires. Cochrane Database Syst Rev. 2007, 2: MR000008-PubMed Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al: Methods to increase response rates to postal questionnaires. Cochrane Database Syst Rev. 2007, 2: MR000008-PubMed
9.
go back to reference Beebe TJ, Davern ME, McAlpine DD, Call KT, Rockwood TH: Increasing response rates in a survey of Medicaid enrollees: the effect of a prepaid monetary incentive and mixed modes (mail and telephone). Med Care. 2005, 43: 411-414. 10.1097/01.mlr.0000156858.81146.0e.CrossRefPubMed Beebe TJ, Davern ME, McAlpine DD, Call KT, Rockwood TH: Increasing response rates in a survey of Medicaid enrollees: the effect of a prepaid monetary incentive and mixed modes (mail and telephone). Med Care. 2005, 43: 411-414. 10.1097/01.mlr.0000156858.81146.0e.CrossRefPubMed
10.
go back to reference Gibson PJ, Koepsell TD, Diehr P, Hale C: Increasing response rates for mailed surveys of Medicaid clients and other low-income populations. Am J Epidemiol. 1999, 149: 1057-1062.CrossRefPubMed Gibson PJ, Koepsell TD, Diehr P, Hale C: Increasing response rates for mailed surveys of Medicaid clients and other low-income populations. Am J Epidemiol. 1999, 149: 1057-1062.CrossRefPubMed
11.
go back to reference Evans BR, Peterson BL, Demark-Wahnefried W: No difference in response rate to a mailed survey among prostate cancer survivors using conditional versus unconditional incentives. Cancer Epidemiol Biomarkers Prev. 2004, 13: 277-278. 10.1158/1055-9965.EPI-03-0065.CrossRefPubMed Evans BR, Peterson BL, Demark-Wahnefried W: No difference in response rate to a mailed survey among prostate cancer survivors using conditional versus unconditional incentives. Cancer Epidemiol Biomarkers Prev. 2004, 13: 277-278. 10.1158/1055-9965.EPI-03-0065.CrossRefPubMed
12.
go back to reference Singer E, van Hoewyk J, Maher MP: Experiments with incentives in telephone surveys. Public Opin Q. 2000, 64: 171-188. 10.1086/317761.CrossRefPubMed Singer E, van Hoewyk J, Maher MP: Experiments with incentives in telephone surveys. Public Opin Q. 2000, 64: 171-188. 10.1086/317761.CrossRefPubMed
13.
go back to reference Mack S, Huggins V, Keathley D, Sundukchi M: Do monetary incentives improve response rates in the survey of income and program participation?. Proceedings of the Section on Survey Research Methods. Alexandria, VA, 529-534. Mack S, Huggins V, Keathley D, Sundukchi M: Do monetary incentives improve response rates in the survey of income and program participation?. Proceedings of the Section on Survey Research Methods. Alexandria, VA, 529-534.
14.
go back to reference Martin E, Abreu D, Winters F: Money and motive: Effects of incentives on panel attrition in the Survey of Income and Program Participation. Journal of Official Statistics. 2001, 17: 284- Martin E, Abreu D, Winters F: Money and motive: Effects of incentives on panel attrition in the Survey of Income and Program Participation. Journal of Official Statistics. 2001, 17: 284-
15.
go back to reference Singer E, Kulka RA: Paying respondents for survey participation. Studies of welfare populations: Data Collection and Research Issues. Edited by: Ver Ploeg M, Moffit RA, Forbes Citro C. 2002, Washington DC: National Academy Press, 105-128. Panel on data and methods for measuring the effects of changes in social welfare programs Singer E, Kulka RA: Paying respondents for survey participation. Studies of welfare populations: Data Collection and Research Issues. Edited by: Ver Ploeg M, Moffit RA, Forbes Citro C. 2002, Washington DC: National Academy Press, 105-128. Panel on data and methods for measuring the effects of changes in social welfare programs
16.
go back to reference Jones R, Zhou M, Yates WR: Improving return rates for health-care outcome. Psychol Rep. 2000, 87: 639-642. 10.2466/PR0.87.6.639-642.CrossRefPubMed Jones R, Zhou M, Yates WR: Improving return rates for health-care outcome. Psychol Rep. 2000, 87: 639-642. 10.2466/PR0.87.6.639-642.CrossRefPubMed
17.
go back to reference Patten SB, Li FX, Cook T, Hilsden RJ, Sutherland LR: Irritable bowel syndrome: are incentives useful for improving survey response rates?. J Clin Epidemiol. 2003, 56: 256-261. 10.1016/S0895-4356(02)00586-3.CrossRefPubMed Patten SB, Li FX, Cook T, Hilsden RJ, Sutherland LR: Irritable bowel syndrome: are incentives useful for improving survey response rates?. J Clin Epidemiol. 2003, 56: 256-261. 10.1016/S0895-4356(02)00586-3.CrossRefPubMed
18.
go back to reference Price JH, Dake JA, Jordan TR, Silvestri KS, Ward BL: Effects of small monetary incentives on return rates of a health survey to adults in rural areas. Psychol Rep. 2006, 98: 849-852. 10.2466/pr0.98.3.849-852.CrossRefPubMed Price JH, Dake JA, Jordan TR, Silvestri KS, Ward BL: Effects of small monetary incentives on return rates of a health survey to adults in rural areas. Psychol Rep. 2006, 98: 849-852. 10.2466/pr0.98.3.849-852.CrossRefPubMed
19.
go back to reference Shaw MJ, Beebe TJ, Jensen HL, Adlis SA: The use of monetary incentives in a community survey: impact on response rates, data quality, and cost. Health Serv Res. 2001, 35: 1339-1346.PubMedPubMedCentral Shaw MJ, Beebe TJ, Jensen HL, Adlis SA: The use of monetary incentives in a community survey: impact on response rates, data quality, and cost. Health Serv Res. 2001, 35: 1339-1346.PubMedPubMedCentral
20.
go back to reference Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Saha S, Snyder A, et al: Variation in Estimates of Limited Health Literacy by Assessment Instruments and Non-Response Bias. JGIM. 2010, 25: 675-681. 10.1007/s11606-010-1304-2.CrossRefPubMedPubMedCentral Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Saha S, Snyder A, et al: Variation in Estimates of Limited Health Literacy by Assessment Instruments and Non-Response Bias. JGIM. 2010, 25: 675-681. 10.1007/s11606-010-1304-2.CrossRefPubMedPubMedCentral
21.
go back to reference Parkes R, Kreiger N, James B, Johnson KC: Effects on subject response of information brochures and small cash incentives in a mail-based case-control study. Ann Epidemiol. 2000, 10: 117-124. 10.1016/S1047-2797(99)00047-2.CrossRefPubMed Parkes R, Kreiger N, James B, Johnson KC: Effects on subject response of information brochures and small cash incentives in a mail-based case-control study. Ann Epidemiol. 2000, 10: 117-124. 10.1016/S1047-2797(99)00047-2.CrossRefPubMed
22.
go back to reference Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, et al: Validation of screening questions for limited health literacy in a large VA outpatient population. Journal of General Internal Medicine. 2008, 23: 561-566. 10.1007/s11606-008-0520-5.CrossRefPubMedPubMedCentral Chew LD, Griffin JM, Partin MR, Noorbaloochi S, Grill JP, Snyder A, et al: Validation of screening questions for limited health literacy in a large VA outpatient population. Journal of General Internal Medicine. 2008, 23: 561-566. 10.1007/s11606-008-0520-5.CrossRefPubMedPubMedCentral
23.
go back to reference The American Association for Public Opinion Research: Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 2009, AAPOR, 6 The American Association for Public Opinion Research: Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 2009, AAPOR, 6
24.
go back to reference Sudman S, Bradburn N: Response Effects in Surveys: A Review and Synthesis. 1974, Chicago: Aldine Pub. Co Sudman S, Bradburn N: Response Effects in Surveys: A Review and Synthesis. 1974, Chicago: Aldine Pub. Co
25.
go back to reference Groves RM, Singer E, Corning A: Leverage-Saliency Theory of Survey Participation: Description and an Illustration. The Public Opinion Quarterly. 2000, 64: 299-308. 10.1086/317990.CrossRefPubMed Groves RM, Singer E, Corning A: Leverage-Saliency Theory of Survey Participation: Description and an Illustration. The Public Opinion Quarterly. 2000, 64: 299-308. 10.1086/317990.CrossRefPubMed
26.
go back to reference Groves RM, Couper MP: Nonresponse in household interview surveys. 1998, New York, NY: WileyCrossRef Groves RM, Couper MP: Nonresponse in household interview surveys. 1998, New York, NY: WileyCrossRef
27.
go back to reference Dillman DA, Smyth J, Christian LM: Internet, Mail and Mixed-Mode Surveys: The Tailored Design Method. 2008, Wiley, 3 Dillman DA, Smyth J, Christian LM: Internet, Mail and Mixed-Mode Surveys: The Tailored Design Method. 2008, Wiley, 3
28.
go back to reference Yammarino FJ, Skinner SJ, Childers TL: Understanding Mail Survey Response Behavior A Meta-Analysis. Public Opin Q. 1991, 55: 613-639. 10.1086/269284.CrossRef Yammarino FJ, Skinner SJ, Childers TL: Understanding Mail Survey Response Behavior A Meta-Analysis. Public Opin Q. 1991, 55: 613-639. 10.1086/269284.CrossRef
29.
go back to reference Groves RM, Dillman DA, Eltinge J, Little RA: Survey Nonresponse. 2001, New York, NY: Wiley Groves RM, Dillman DA, Eltinge J, Little RA: Survey Nonresponse. 2001, New York, NY: Wiley
30.
go back to reference Alexander GL, Divine GW, Couper MP, McClure JB, Stopponi MA, Fortman KK, et al: Effect of Incentives and Mailing Features on Online Health Program Enrollment. American Journal of Preventive Medicine. 2008, 34: 382-388. 10.1016/j.amepre.2008.01.028.CrossRefPubMedPubMedCentral Alexander GL, Divine GW, Couper MP, McClure JB, Stopponi MA, Fortman KK, et al: Effect of Incentives and Mailing Features on Online Health Program Enrollment. American Journal of Preventive Medicine. 2008, 34: 382-388. 10.1016/j.amepre.2008.01.028.CrossRefPubMedPubMedCentral
Metadata
Title
A comparison of small monetary incentives to convert survey non-respondents: a randomized control trial
Authors
Joan M Griffin
Alisha Baines Simon
Erin Hulbert
John Stevenson
Joseph P Grill
Siamak Noorbaloochi
Melissa R Partin
Publication date
01-12-2011
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2011
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-11-81

Other articles of this Issue 1/2011

BMC Medical Research Methodology 1/2011 Go to the issue