Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Research article

Practical application of opt-out recruitment methods in two health services research studies

Authors: Christopher J. Miller, James F. Burgess Jr., Ellen P. Fischer, Deborah J. Hodges, Lindsay K. Belanger, Jessica M. Lipschitz, Siena R. Easley, Christopher J. Koenig, Regina L. Stanley, Jeffrey M. Pyne

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

Participant recruitment is an ongoing challenge in health research. Recruitment may be especially difficult for studies of access to health care because, even among those who are in care, people using services least often also may be hardest to contact and recruit. Opt-out recruitment methods (in which potential participants are given the opportunity to decline further contact about the study (opt out) following an initial mailing, and are then contacted directly if they have not opted out within a specified period) can be used for such studies. However, there is a dearth of literature on the effort needed for effective opt-out recruitment.

Methods

In this paper we describe opt-out recruitment procedures for two studies on access to health care within the U.S. Department of Veterans Affairs. We report resource requirements for recruitment efforts (number of opt-out packets mailed and number of phone calls made). We also compare the characteristics of study participants to potential participants via t-tests, Fisher’s exact tests, and chi-squared tests.

Results

Recruitment rates for our two studies were 12 and 21%, respectively. Across multiple study sites, we had to send between 4.3 and 9.2 opt-out packets to recruit one participant. The number of phone calls required to arrive at a final status for each potentially eligible Veteran (i.e. study participation or the termination of recruitment efforts) were 2.9 and 6.1 in the two studies, respectively. Study participants differed as expected from the population of potentially eligible Veterans based on planned oversampling of certain subpopulations. The final samples of participants did not differ statistically from those who were mailed opt-out packets, with one exception: in one of our two studies, participants had higher rates of mental health service use in the past year than did those mailed opt-out packets (64 vs. 47%).

Conclusions

Our results emphasize the practicality of using opt-out methods for studies of access to health care. Despite the benefits of these methods, opt-out alone may be insufficient to eliminate non-response bias on key variables. Researchers will need to balance considerations of sample representativeness and feasibility when designing studies investigating access to care.
Appendix
Available only for authorised users
Literature
1.
go back to reference Johnson LC, Beaton R, Murphy S, Pike K. Sampling bias and other methodological threats to the validity of health survey research. Int J Stress Mgmt. 2000;7(4):247–67.CrossRef Johnson LC, Beaton R, Murphy S, Pike K. Sampling bias and other methodological threats to the validity of health survey research. Int J Stress Mgmt. 2000;7(4):247–67.CrossRef
2.
go back to reference Treweek S, Pitkethly M, Cook J, Kjeldstrom M, Taskila T, Johansen M, Mitchell E. Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev. 2011;10-.[MR000013]. doi:10.1002/14651858.MR000013.pub5. Treweek S, Pitkethly M, Cook J, Kjeldstrom M, Taskila T, Johansen M, Mitchell E. Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev. 2011;10-.[MR000013]. doi:10.​1002/​14651858.​MR000013.​pub5.
3.
go back to reference Sadler GR, Lee HC, Lim RSH, Fullerton J. Recruitment of hard‐to‐reach population subgroups via adaptations of the snowball sampling strategy. Nurs Health Sci. 2010;12(3):369–74.CrossRefPubMedPubMedCentral Sadler GR, Lee HC, Lim RSH, Fullerton J. Recruitment of hard‐to‐reach population subgroups via adaptations of the snowball sampling strategy. Nurs Health Sci. 2010;12(3):369–74.CrossRefPubMedPubMedCentral
4.
go back to reference Atkinson R, Flint J. Accessing hidden and hard-to-reach populations: snowball research strategies. Soc Res Update. 2001;33(1):1–4. Atkinson R, Flint J. Accessing hidden and hard-to-reach populations: snowball research strategies. Soc Res Update. 2001;33(1):1–4.
5.
go back to reference Salganik MJ. Variance estimation, design effects, and sample size calculations for respondent-driven sampling. J Urban Health. 2006;83(1):98–112.CrossRefPubMedCentral Salganik MJ. Variance estimation, design effects, and sample size calculations for respondent-driven sampling. J Urban Health. 2006;83(1):98–112.CrossRefPubMedCentral
6.
go back to reference Heckathorn DD. Respondent-driven sampling: a new approach to the study of hidden populations. Soc Probl. 1997;44:174–99.CrossRef Heckathorn DD. Respondent-driven sampling: a new approach to the study of hidden populations. Soc Probl. 1997;44:174–99.CrossRef
7.
go back to reference Heckathorn DD. Respondent-driven sampling II: deriving valid population estimates from chain-referral samples of hidden populations. Soc Probl. 2002;49(1):11–34.CrossRef Heckathorn DD. Respondent-driven sampling II: deriving valid population estimates from chain-referral samples of hidden populations. Soc Probl. 2002;49(1):11–34.CrossRef
8.
go back to reference Funkhouser E, Macaluso M, Wang X. Alternative strategies for selecting population controls: comparison of random digit dialing and targeted telephone calls. Ann Epidemiol. 2000;10(1):59–67.CrossRefPubMed Funkhouser E, Macaluso M, Wang X. Alternative strategies for selecting population controls: comparison of random digit dialing and targeted telephone calls. Ann Epidemiol. 2000;10(1):59–67.CrossRefPubMed
9.
go back to reference Annas GJ. HIPAA regulations-a new era of medical-record privacy? N Engl J Med. 2003;348(15):1486–90.CrossRefPubMed Annas GJ. HIPAA regulations-a new era of medical-record privacy? N Engl J Med. 2003;348(15):1486–90.CrossRefPubMed
10.
go back to reference Redsell SA, Cheater FM. The data protection act (1998): implications for health researchers. J Adv Nurs. 2001;35(4):508–13.CrossRefPubMed Redsell SA, Cheater FM. The data protection act (1998): implications for health researchers. J Adv Nurs. 2001;35(4):508–13.CrossRefPubMed
11.
go back to reference Junghans C, Feder G, Hemingway H, Timmis A, Jones M. Recruiting patients to medical research: double blind randomised trial of “opt-in” versus “opt-out” strategies. BMJ. 2005;331(7522):940.CrossRefPubMedPubMedCentral Junghans C, Feder G, Hemingway H, Timmis A, Jones M. Recruiting patients to medical research: double blind randomised trial of “opt-in” versus “opt-out” strategies. BMJ. 2005;331(7522):940.CrossRefPubMedPubMedCentral
12.
go back to reference Hunt KJ, Shlomo N, Addington-Hall J. Participant recruitment in sensitive surveys: a comparative trial of ‘opt in’versus ‘opt out’approaches. BMC Med Res Methodol. 2013;13(1):3.CrossRefPubMedPubMedCentral Hunt KJ, Shlomo N, Addington-Hall J. Participant recruitment in sensitive surveys: a comparative trial of ‘opt in’versus ‘opt out’approaches. BMC Med Res Methodol. 2013;13(1):3.CrossRefPubMedPubMedCentral
13.
go back to reference Krousel-Wood M, Muntner P, Jannu A, Hyre A, Breault J. Does waiver of written informed consent from the institutional review board affect response rate in a Low‐risk research study? J Investig Med. 2006;54(4):174–9.CrossRefPubMed Krousel-Wood M, Muntner P, Jannu A, Hyre A, Breault J. Does waiver of written informed consent from the institutional review board affect response rate in a Low‐risk research study? J Investig Med. 2006;54(4):174–9.CrossRefPubMed
14.
go back to reference Kaufman D, Bollinger J, Dvoskin R, Scott J. Preferences for opt-in and opt-out enrollment and consent models in biobank research: a national survey of Veterans Administration patients. Genet Med. 2012;14(9):787–94.CrossRefPubMed Kaufman D, Bollinger J, Dvoskin R, Scott J. Preferences for opt-in and opt-out enrollment and consent models in biobank research: a national survey of Veterans Administration patients. Genet Med. 2012;14(9):787–94.CrossRefPubMed
15.
go back to reference Schroy PC, Glick JT, Robinson P, Lydotes MA, Heeren TC, Prout M, Davidson P, Wong JB. A cost-effectiveness analysis of subject recruitment strategies in the HIPAA era: results from a colorectal cancer screening adherence trial. Clin Trials. 2009;6(6):597–609.CrossRefPubMed Schroy PC, Glick JT, Robinson P, Lydotes MA, Heeren TC, Prout M, Davidson P, Wong JB. A cost-effectiveness analysis of subject recruitment strategies in the HIPAA era: results from a colorectal cancer screening adherence trial. Clin Trials. 2009;6(6):597–609.CrossRefPubMed
16.
go back to reference Bray I, Noble S, Boyd A, Brown L, Hayes P, Malcolm J, Robinson R, Williams R, Burston K, Macleod J. A randomised controlled trial comparing opt-in and opt-out home visits for tracing lost participants in a prospective birth cohort study. BMC Med Res Methodol. 2015;15(1):52.CrossRefPubMedPubMedCentral Bray I, Noble S, Boyd A, Brown L, Hayes P, Malcolm J, Robinson R, Williams R, Burston K, Macleod J. A randomised controlled trial comparing opt-in and opt-out home visits for tracing lost participants in a prospective birth cohort study. BMC Med Res Methodol. 2015;15(1):52.CrossRefPubMedPubMedCentral
17.
go back to reference Boland J, Currow DC, Wilcock A, Tieman J, Hussain JA, Pitsillides C, Abernethy AP, Johnson MJ. A systematic review of strategies used to increase recruitment of people with cancer or organ failure into clinical trials: implications for palliative care research. J Pain Symptom Manage. 2015;49(4):762–72. e765.CrossRefPubMed Boland J, Currow DC, Wilcock A, Tieman J, Hussain JA, Pitsillides C, Abernethy AP, Johnson MJ. A systematic review of strategies used to increase recruitment of people with cancer or organ failure into clinical trials: implications for palliative care research. J Pain Symptom Manage. 2015;49(4):762–72. e765.CrossRefPubMed
18.
go back to reference Parry O, Bancroft A, Gnich W, Amos A. Nobody home? issues of respondent recruitment in areas of deprivation. Critical Public Health. 2001;11(4):305–17.CrossRef Parry O, Bancroft A, Gnich W, Amos A. Nobody home? issues of respondent recruitment in areas of deprivation. Critical Public Health. 2001;11(4):305–17.CrossRef
19.
go back to reference Nattinger AB, Pezzin LE, Sparapani RA, Neuner JM, King TK, Laud PW. Heightened attention to medical privacy: challenges for unbiased sample recruitment and a possible solution. Am J Epidemiol. 2010;172(6):637–44.CrossRefPubMedPubMedCentral Nattinger AB, Pezzin LE, Sparapani RA, Neuner JM, King TK, Laud PW. Heightened attention to medical privacy: challenges for unbiased sample recruitment and a possible solution. Am J Epidemiol. 2010;172(6):637–44.CrossRefPubMedPubMedCentral
20.
22.
go back to reference Kahneman D, Tversky A. Prospect theory: An analysis of decision under risk. Econometrica: J Economet Soc. 1979;47(2):263–91. Kahneman D, Tversky A. Prospect theory: An analysis of decision under risk. Econometrica: J Economet Soc. 1979;47(2):263–91.
23.
go back to reference Tversky A, Kahneman D. Loss aversion in riskless choice: A reference-dependent model. Q J Econ. 1991;1039–1061. Tversky A, Kahneman D. Loss aversion in riskless choice: A reference-dependent model. Q J Econ. 1991;1039–1061.
24.
go back to reference Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17(9):643–53.CrossRefPubMed Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17(9):643–53.CrossRefPubMed
25.
go back to reference Bernard HR, Ryan GW. Analyzing qualitative data: Systematic approaches. Washington: SAGE publications; 2009. Bernard HR, Ryan GW. Analyzing qualitative data: Systematic approaches. Washington: SAGE publications; 2009.
26.
go back to reference Fear NT, Van Staden L, Iversen A, Hall J, Wessely S. Fifty ways to trace your veteran: increasing response rates can be cheap and effective. Eur J Psychotraum. 2010;1:5516.CrossRef Fear NT, Van Staden L, Iversen A, Hall J, Wessely S. Fifty ways to trace your veteran: increasing response rates can be cheap and effective. Eur J Psychotraum. 2010;1:5516.CrossRef
27.
go back to reference Breiding MJ. Prevalence and characteristics of sexual violence, stalking, and intimate partner violence victimization—national intimate partner and sexual violence survey, united states, 2011. Morb Mortal Wkly Rep Surveill Summ (Washington, DC: 2002). 2014;63(8):1. Breiding MJ. Prevalence and characteristics of sexual violence, stalking, and intimate partner violence victimization—national intimate partner and sexual violence survey, united states, 2011. Morb Mortal Wkly Rep Surveill Summ (Washington, DC: 2002). 2014;63(8):1.
28.
go back to reference Welte J, Barnes G, Wieczorek W, Tidwell M-C, Parker J. Alcohol and gambling pathology among US adults: prevalence, demographic patterns and comorbidity. J Stud Alcohol. 2001;62(5):706–12.CrossRefPubMed Welte J, Barnes G, Wieczorek W, Tidwell M-C, Parker J. Alcohol and gambling pathology among US adults: prevalence, demographic patterns and comorbidity. J Stud Alcohol. 2001;62(5):706–12.CrossRefPubMed
29.
go back to reference Shannon CA, Rospenda KM, Richman JA, Minich LM. Race, racial discrimination, and the risk of work-related illness, injury or assault: findings from a national study. J Occup Env Med. 2009;51(4):441.CrossRef Shannon CA, Rospenda KM, Richman JA, Minich LM. Race, racial discrimination, and the risk of work-related illness, injury or assault: findings from a national study. J Occup Env Med. 2009;51(4):441.CrossRef
30.
go back to reference Haga SB, Kawamoto K, Agans R, Ginsburg GS. Consideration of patient preferences and challenges in storage and access of pharmacogenetic test results. Genet Med. 2011;13(10):887.CrossRefPubMedPubMedCentral Haga SB, Kawamoto K, Agans R, Ginsburg GS. Consideration of patient preferences and challenges in storage and access of pharmacogenetic test results. Genet Med. 2011;13(10):887.CrossRefPubMedPubMedCentral
31.
go back to reference Maeng DD, Martsolf GR, Scanlon DP, Christianson JB. Care coordination for the chronically ill: understanding the patient’s perspective. Health Serv Res. 2012;47(5):1960–79.CrossRefPubMedPubMedCentral Maeng DD, Martsolf GR, Scanlon DP, Christianson JB. Care coordination for the chronically ill: understanding the patient’s perspective. Health Serv Res. 2012;47(5):1960–79.CrossRefPubMedPubMedCentral
32.
go back to reference Elkington J, Stevenson M, Haworth N, Sharwood L. Using police crash databases for injury prevention research–a comparison of opt‐out and opt‐in approaches to study recruitment. Aust N Z J Public Health. 2014;38(3):286–9.CrossRefPubMed Elkington J, Stevenson M, Haworth N, Sharwood L. Using police crash databases for injury prevention research–a comparison of opt‐out and opt‐in approaches to study recruitment. Aust N Z J Public Health. 2014;38(3):286–9.CrossRefPubMed
33.
go back to reference Sygna K, Johansen S, Ruland CM. Recruitment challenges in clinical research including cancer patients and caregivers. Trials. 2015;16(1):1.CrossRef Sygna K, Johansen S, Ruland CM. Recruitment challenges in clinical research including cancer patients and caregivers. Trials. 2015;16(1):1.CrossRef
34.
go back to reference Trevena L, Irwig L, Barratt A. Impact of privacy legislation on the number and characteristics of people who are recruited for research: a randomised controlled trial. J Med Ethics. 2006;32(8):473–7.CrossRefPubMedPubMedCentral Trevena L, Irwig L, Barratt A. Impact of privacy legislation on the number and characteristics of people who are recruited for research: a randomised controlled trial. J Med Ethics. 2006;32(8):473–7.CrossRefPubMedPubMedCentral
35.
go back to reference Goodman A, Gatward R. Who are we missing? area deprivation and survey participation. Eur J Epidemiol. 2008;23(6):379–87.CrossRefPubMed Goodman A, Gatward R. Who are we missing? area deprivation and survey participation. Eur J Epidemiol. 2008;23(6):379–87.CrossRefPubMed
36.
go back to reference Vellinga A, Cormican M, Hanahoe B, Bennett K, Murphy AW. Opt-out as an acceptable method of obtaining consent in medical research: a short report. BMC Med Res Methodol. 2011;11(1):40.CrossRefPubMedPubMedCentral Vellinga A, Cormican M, Hanahoe B, Bennett K, Murphy AW. Opt-out as an acceptable method of obtaining consent in medical research: a short report. BMC Med Res Methodol. 2011;11(1):40.CrossRefPubMedPubMedCentral
37.
go back to reference Dunn KM, Jordan K, Lacey RJ, Shapley M, Jinks C. Patterns of consent in epidemiologic research: evidence from over 25,000 responders. Am J Epidemiol. 2004;159(11):1087–94.CrossRefPubMed Dunn KM, Jordan K, Lacey RJ, Shapley M, Jinks C. Patterns of consent in epidemiologic research: evidence from over 25,000 responders. Am J Epidemiol. 2004;159(11):1087–94.CrossRefPubMed
38.
go back to reference O’Neil MJ. Estimating the nonresponse BiasDue to refusals in telephone surveys. Public Opin Q. 1979;43(2):218–32.CrossRef O’Neil MJ. Estimating the nonresponse BiasDue to refusals in telephone surveys. Public Opin Q. 1979;43(2):218–32.CrossRef
39.
go back to reference Newington L, Metcalfe A. Factors influencing recruitment to research: qualitative study of the experiences and perceptions of research teams. BMC Med Res Methodol. 2014;14(1):1.CrossRef Newington L, Metcalfe A. Factors influencing recruitment to research: qualitative study of the experiences and perceptions of research teams. BMC Med Res Methodol. 2014;14(1):1.CrossRef
Metadata
Title
Practical application of opt-out recruitment methods in two health services research studies
Authors
Christopher J. Miller
James F. Burgess Jr.
Ellen P. Fischer
Deborah J. Hodges
Lindsay K. Belanger
Jessica M. Lipschitz
Siena R. Easley
Christopher J. Koenig
Regina L. Stanley
Jeffrey M. Pyne
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-017-0333-5

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue