Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Research article

Using Mechanical Turk to recruit participants for internet intervention research: experience from recruitment for four trials targeting hazardous alcohol consumption

Authors: John A. Cunningham, Alexandra Godinho, Vladyslav Kushnir

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

Mechanical Turk (MTurk) is an online portal operated by Amazon where ‘requesters’ (individuals or businesses) can submit jobs for ‘workers.’ MTurk is used extensively by academics as a quick and cheap means of collecting questionnaire data, including information on alcohol consumption, from a diverse sample of participants. We tested the feasibility of recruiting for alcohol Internet intervention trials through MTurk.

Methods

Participants, 18 years or older, who drank at least weekly were recruited for four intervention trials (combined sample size, N = 11,107). The same basic recruitment strategy was employed for each trial – invite participants to complete a survey about alcohol consumption (less than 15 min in length, US$1.50 payment), identify eligible participants who drank in a hazardous fashion, invite those eligible to complete a follow-up survey ($10 payment), randomize participants to be sent or not sent information to access an online intervention for hazardous alcohol use. Procedures where put in place to optimize the chances that participants could only complete the baseline survey once.

Results

There was a substantially slower rate of recruitment by the fourth trial compared to the earlier trials. Demographic characteristics also varied across trials (age, sex, employment and marital status). Patterns of alcohol consumption, while displaying some differences, did not appear to vary in a linear fashion between trials.

Conclusions

It is possible to recruit large (but not inexhaustible) numbers of people who drink in a hazardous fashion. Issues for online intervention research when employing this sample are discussed.
Literature
1.
go back to reference Gross MS, Liu NH, Contreras O, Munoz RF, Leykin Y. Using Google AdWords for international multilingual recruitment to health research websites. J Med Internet Res. 2014;16(1):e18.CrossRefPubMedPubMedCentral Gross MS, Liu NH, Contreras O, Munoz RF, Leykin Y. Using Google AdWords for international multilingual recruitment to health research websites. J Med Internet Res. 2014;16(1):e18.CrossRefPubMedPubMedCentral
2.
go back to reference Thornton LK, Harris K, Baker AL, Johnson M, Kay-Lambkin FJ. Recruiting for addiction research via Facebook. Drug Alcohol Rev. 2016;35(4):494–502.CrossRefPubMed Thornton LK, Harris K, Baker AL, Johnson M, Kay-Lambkin FJ. Recruiting for addiction research via Facebook. Drug Alcohol Rev. 2016;35(4):494–502.CrossRefPubMed
3.
go back to reference Whitaker C, Stevelink S, Fear N. The use of Facebook in recruiting participants for Health Research purposes: a systematic review. J Med Internet Res. 2017;19(8):e290.CrossRefPubMedPubMedCentral Whitaker C, Stevelink S, Fear N. The use of Facebook in recruiting participants for Health Research purposes: a systematic review. J Med Internet Res. 2017;19(8):e290.CrossRefPubMedPubMedCentral
4.
go back to reference Shapiro D, Chandler J, Mueller PA. Using mechanical Turk to study clinical populations. Clin Psychol Sci. 2014;1:213–20.CrossRef Shapiro D, Chandler J, Mueller PA. Using mechanical Turk to study clinical populations. Clin Psychol Sci. 2014;1:213–20.CrossRef
5.
go back to reference Buhrmester M, Kwang T, Gosling SD. Amazon's mechanical Turk: a new source of inexpensive, yet high-quality, data? Perspect Psychol Sci. 2011;6(1):3–5.CrossRefPubMed Buhrmester M, Kwang T, Gosling SD. Amazon's mechanical Turk: a new source of inexpensive, yet high-quality, data? Perspect Psychol Sci. 2011;6(1):3–5.CrossRefPubMed
6.
go back to reference Chandler J, Shapiro D. Conducting clinical research using Crowdsourced convenience samples. Annu Rev Clin Psychol. 2016;12:53–81.CrossRefPubMed Chandler J, Shapiro D. Conducting clinical research using Crowdsourced convenience samples. Annu Rev Clin Psychol. 2016;12:53–81.CrossRefPubMed
7.
go back to reference Daly TM, Nataraajan R. Swapping bricks for clicks: crowdsourcing longitudinal data on Amazon Turk. J Business Res. 2015;68:2603–9.CrossRef Daly TM, Nataraajan R. Swapping bricks for clicks: crowdsourcing longitudinal data on Amazon Turk. J Business Res. 2015;68:2603–9.CrossRef
8.
go back to reference Litman L, Robinson J, Abberbock T. TurkPrime.Com: a versatile crowdsourcing data acquisition platform for the behavioral sciences. Behav Res Methods. 2017;49(2):433–442. Litman L, Robinson J, Abberbock T. TurkPrime.Com: a versatile crowdsourcing data acquisition platform for the behavioral sciences. Behav Res Methods. 2017;49(2):433–442.
9.
go back to reference Wiens TK, Walker LJ. The chronic disease concept of addiction: helpful or harmful? Addict Res Theory. 2015;23:309–21.CrossRef Wiens TK, Walker LJ. The chronic disease concept of addiction: helpful or harmful? Addict Res Theory. 2015;23:309–21.CrossRef
10.
go back to reference Kim HS, Hodgins DC. Reliability and validity of data obtained from alcohol, cannabis, and gambling populations on Amazon's mechanical Turk. Psychol Addict Behav. 2017;31(1):85–94.CrossRefPubMed Kim HS, Hodgins DC. Reliability and validity of data obtained from alcohol, cannabis, and gambling populations on Amazon's mechanical Turk. Psychol Addict Behav. 2017;31(1):85–94.CrossRefPubMed
11.
go back to reference Bunge EL, Williamson RE, Cano M, Leykin Y, Munoz RF. Mood management effects of brief unsupported internet interventions. Internet Interv. 2016;5:36–43.CrossRef Bunge EL, Williamson RE, Cano M, Leykin Y, Munoz RF. Mood management effects of brief unsupported internet interventions. Internet Interv. 2016;5:36–43.CrossRef
12.
go back to reference Kristan J, Suffoletto B. Using online crowdsourcing to understand young adult attitudes toward expert-authored messages aimed at reducing hazardous alcohol consumption and to collect peer-authored messages. Transl Behav Med. 2015;5(1):45–52.CrossRefPubMed Kristan J, Suffoletto B. Using online crowdsourcing to understand young adult attitudes toward expert-authored messages aimed at reducing hazardous alcohol consumption and to collect peer-authored messages. Transl Behav Med. 2015;5(1):45–52.CrossRefPubMed
13.
go back to reference Peer E, Vosgerau J, Acquisti A. Reputation as a sufficient condition for data quality on Amazon mechanical Turk. Behav Res Methods. 2014;46(4):1023–31.CrossRefPubMed Peer E, Vosgerau J, Acquisti A. Reputation as a sufficient condition for data quality on Amazon mechanical Turk. Behav Res Methods. 2014;46(4):1023–31.CrossRefPubMed
15.
go back to reference Wechsler H, Davenport A, Dowdall G, Moeykens B, Castillo S. Health and behavioral consequences of binge drinking in college: a national survey of students at 140 campuses. JAMA. 1994;272:1672–7.CrossRefPubMed Wechsler H, Davenport A, Dowdall G, Moeykens B, Castillo S. Health and behavioral consequences of binge drinking in college: a national survey of students at 140 campuses. JAMA. 1994;272:1672–7.CrossRefPubMed
16.
go back to reference Grant BF, Moore TC, Shepard J, Kaplan K. Source and accuracy statement. Wave 1. National Epidemiologic Survey on alcohol and related conditions (NESARC). Bethesda: National Institute on Alcohol Abuse and Alcoholism; 2003. Grant BF, Moore TC, Shepard J, Kaplan K. Source and accuracy statement. Wave 1. National Epidemiologic Survey on alcohol and related conditions (NESARC). Bethesda: National Institute on Alcohol Abuse and Alcoholism; 2003.
17.
go back to reference Cunningham JA, Godinho A, Kushnir V. Can Amazon's mechanical Turk be used to recruit participants for internet intervention trials? A pilot study involving a randomized controlled trial of a brief online intervention for hazardous alcohol use. Internet Interv. 2017;10:12–6.CrossRef Cunningham JA, Godinho A, Kushnir V. Can Amazon's mechanical Turk be used to recruit participants for internet intervention trials? A pilot study involving a randomized controlled trial of a brief online intervention for hazardous alcohol use. Internet Interv. 2017;10:12–6.CrossRef
18.
go back to reference Cunningham JA, Wild TC, Cordingley J, van Mierlo T, Humphreys K. A randomized controlled trial of an internet-based intervention for alcohol abusers. Addiction. 2009;104(12):2023–32.CrossRefPubMedPubMedCentral Cunningham JA, Wild TC, Cordingley J, van Mierlo T, Humphreys K. A randomized controlled trial of an internet-based intervention for alcohol abusers. Addiction. 2009;104(12):2023–32.CrossRefPubMedPubMedCentral
19.
20.
go back to reference Cunningham JA, Shorter GW, Murphy M, Kushnir V, Rehm J, Hendershot CS. Randomized controlled trial of a brief versus extended internet intervention for problem drinkers. Int J Behav Med. 2017;5:760–7.CrossRef Cunningham JA, Shorter GW, Murphy M, Kushnir V, Rehm J, Hendershot CS. Randomized controlled trial of a brief versus extended internet intervention for problem drinkers. Int J Behav Med. 2017;5:760–7.CrossRef
21.
go back to reference Riper H, Blankers M, Hadiwijaya H, Cunningham J, Clarke S, Wiers R, Ebert D, Cuijpers P. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One. 2014;9(6):e99912.CrossRefPubMedPubMedCentral Riper H, Blankers M, Hadiwijaya H, Cunningham J, Clarke S, Wiers R, Ebert D, Cuijpers P. Effectiveness of guided and unguided low-intensity internet interventions for adult alcohol misuse: a meta-analysis. PLoS One. 2014;9(6):e99912.CrossRefPubMedPubMedCentral
22.
go back to reference Trewin D, Lee G. International comparisons of telephone coverage. In: Groves RM, Biemer PP, Lyberg LE, Massey W, Nicholls L, Waksberg J, editors. Telephone survey methodology. New York: Wiley; 1988. p. 9–24. Trewin D, Lee G. International comparisons of telephone coverage. In: Groves RM, Biemer PP, Lyberg LE, Massey W, Nicholls L, Waksberg J, editors. Telephone survey methodology. New York: Wiley; 1988. p. 9–24.
23.
go back to reference Kandel D. The social demography of drug use. Milbank Quarterly. 1991;69(3):365–414. Kandel D. The social demography of drug use. Milbank Quarterly. 1991;69(3):365–414.
24.
go back to reference Babor TF, Steinberg K, Anton R. Talk is cheap: measuring drinking outcomes in clinical trials. J Stud Alcohol. 2000;61:55–63.CrossRefPubMed Babor TF, Steinberg K, Anton R. Talk is cheap: measuring drinking outcomes in clinical trials. J Stud Alcohol. 2000;61:55–63.CrossRefPubMed
25.
go back to reference Cunningham JA, Koski-Jannes A, Wild TC, Cordingley J. Treating alcohol problems with self-help materials: a population study. J Stud Alcohol. 2002;63(6):649–54.CrossRefPubMed Cunningham JA, Koski-Jannes A, Wild TC, Cordingley J. Treating alcohol problems with self-help materials: a population study. J Stud Alcohol. 2002;63(6):649–54.CrossRefPubMed
Metadata
Title
Using Mechanical Turk to recruit participants for internet intervention research: experience from recruitment for four trials targeting hazardous alcohol consumption
Authors
John A. Cunningham
Alexandra Godinho
Vladyslav Kushnir
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-017-0440-3

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue