Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2018

Open Access 01-12-2018 | Research article

List randomization for eliciting HIV status and sexual behaviors in rural KwaZulu-Natal, South Africa: a randomized experiment using known true values for validation

Authors: Noah Haber, Guy Harling, Jessica Cohen, Tinofa Mutevedzi, Frank Tanser, Dickman Gareta, Kobus Herbst, Deenan Pillay, Till Bärnighausen, Günther Fink

Published in: BMC Medical Research Methodology | Issue 1/2018

Login to get access

Abstract

Background

List randomization (LR), a survey method intended to mitigate biases related to sensitive true/false questions, has received recent attention from researchers. However, tests of its validity are limited, with no study comparing LR-elicited results with individually known truths. We conducted a test of LR for HIV-related responses in a high HIV prevalence setting in KwaZulu-Natal. By using researcher-known HIV serostatus and HIV test refusal data, we were able to assess how LR and direct questionnaires perform against individual known truth.

Methods

Participants were recruited from the participation list from the 2016 round of the Africa Health Research Institute demographic surveillance system, oversampling individuals who were HIV positive. Participants were randomized to two study arms. In Arm A, participants were presented five true/false statements, one of which was the sensitive item, the others non-sensitive. Participants were then asked how many of the five statements they believed were true. In Arm B, participants were asked about each statement individually. LR estimates used data from both arms, while direct estimates were generated from Arm B alone. We compared elicited responses to HIV testing and serostatus data collected through the demographic surveillance system.

Results

We enrolled 483 participants, 262 (54%) were randomly assigned to Arm A, and 221 (46%) to Arm B. LR estimated 56% (95% CI: 40 to 72%) of the population to be HIV-negative, compared to 47% (95% CI: 39 to 54%) using direct estimates; the population-estimate of the true value was 32% (95% CI: 28 to 36%). LR estimates yielded HIV test refusal percentages of 55% (95% CI: 37 to 73%) compared to 13% (95% CI: 8 to 17%) by direct estimation, and 15% (95% CI: 12 to 18%) based on observed past behavior.

Conclusions

In this context, LR performed poorly when compared to known truth, and did not improve estimates over direct questioning methods when comparing with known truth. These results may reflect difficulties in implementation or comprehension of the LR approach, which is inherently complex. Adjustments to delivery procedures may improve LR’s usefulness. Further investigation of the cognitive processes of participants in answering LR surveys is warranted.
Appendix
Available only for authorised users
Literature
1.
go back to reference Smith D. Correcting for social desirability response sets in opinion-attitude survey research. Public Opinion Q. 1967;31(1):87–94.CrossRef Smith D. Correcting for social desirability response sets in opinion-attitude survey research. Public Opinion Q. 1967;31(1):87–94.CrossRef
3.
go back to reference Aho J, Koushik A, Diakite SL, Loua KM, Nguyen VK, Rashed S. Biological validation of self-reported condom use among sex workers in Guinea. AIDS Behav. 2010;14(6):1287–93.CrossRefPubMed Aho J, Koushik A, Diakite SL, Loua KM, Nguyen VK, Rashed S. Biological validation of self-reported condom use among sex workers in Guinea. AIDS Behav. 2010;14(6):1287–93.CrossRefPubMed
4.
go back to reference Diclemente R, Swartzendruber AL, Brown J. Improving the validity of self-reported sexual behavior: no easy answers. Sex Transmit Dis. 2013;40(2):111–2.CrossRef Diclemente R, Swartzendruber AL, Brown J. Improving the validity of self-reported sexual behavior: no easy answers. Sex Transmit Dis. 2013;40(2):111–2.CrossRef
5.
go back to reference Kelly CA, Soler-Hampejsek E, Mensch BS, Hewett PC. Social desirability bias in sexual behavior reporting: evidence from an interview mode experiment in rural Malawi. Int Perspect Sex Reproduct Health. 2013;39(1):14–21.CrossRef Kelly CA, Soler-Hampejsek E, Mensch BS, Hewett PC. Social desirability bias in sexual behavior reporting: evidence from an interview mode experiment in rural Malawi. Int Perspect Sex Reproduct Health. 2013;39(1):14–21.CrossRef
6.
go back to reference Ahart A, Sackett P. A new method of examining relationships between individual difference measures and sensitive behavior criteria: evaluating the unmatched count technique. Organizational Res Methods. 2004;7(1):101–14.CrossRef Ahart A, Sackett P. A new method of examining relationships between individual difference measures and sensitive behavior criteria: evaluating the unmatched count technique. Organizational Res Methods. 2004;7(1):101–14.CrossRef
7.
go back to reference Blair G, Imai K. Statistical analysis of list experiments. Polit Anal. 2012;20(1):47–77.CrossRef Blair G, Imai K. Statistical analysis of list experiments. Polit Anal. 2012;20(1):47–77.CrossRef
8.
go back to reference Imai K. Multivariate regression analysis for the item count technique. J Am Statist Assoc. 2011;106(494):407–16.CrossRef Imai K. Multivariate regression analysis for the item count technique. J Am Statist Assoc. 2011;106(494):407–16.CrossRef
9.
go back to reference Kuha J, Jackson J. The item count method for sensitive survey questions: modelling criminal behaviour. J Royal Stat Soc Series C (Applied Statistics). 2014;63(2):321–41.CrossRef Kuha J, Jackson J. The item count method for sensitive survey questions: modelling criminal behaviour. J Royal Stat Soc Series C (Applied Statistics). 2014;63(2):321–41.CrossRef
10.
go back to reference LaBrie J, Earleywine M. Sexual risk behaviors and alcohol: higher base rates revealed using the unmatched count technique. J Sex Res. 2000;37(4):321–6.CrossRef LaBrie J, Earleywine M. Sexual risk behaviors and alcohol: higher base rates revealed using the unmatched count technique. J Sex Res. 2000;37(4):321–6.CrossRef
11.
go back to reference Walsh JA, Braithwaite J. Self-reported alcohol consumption and sexual behavior in males and females: using the unmatched-count technique to examine reporting practices of socially sensitive subjects in a sample of university students. J Alcohol Drug Educ. 2008;52(2):49–72. Walsh JA, Braithwaite J. Self-reported alcohol consumption and sexual behavior in males and females: using the unmatched-count technique to examine reporting practices of socially sensitive subjects in a sample of university students. J Alcohol Drug Educ. 2008;52(2):49–72.
12.
go back to reference Starosta AJ, Earleywine M. Assessing base rates of sexual behavior using the unmatched count technique. Health Psychol Behav Med Open Access J. 2014;2(1):198–210.CrossRef Starosta AJ, Earleywine M. Assessing base rates of sexual behavior using the unmatched count technique. Health Psychol Behav Med Open Access J. 2014;2(1):198–210.CrossRef
13.
go back to reference Arentoft A, Van Dyk K, Thames AD, Sayegh P, Thaler N, Schonfeld D, LaBrie J, Hinkin CH. Comparing the unmatched count technique and direct self-report for sensitive health-risk behaviors in HIV+ adults. AIDS Care. 2016;28(3):370–5.CrossRefPubMed Arentoft A, Van Dyk K, Thames AD, Sayegh P, Thaler N, Schonfeld D, LaBrie J, Hinkin CH. Comparing the unmatched count technique and direct self-report for sensitive health-risk behaviors in HIV+ adults. AIDS Care. 2016;28(3):370–5.CrossRefPubMed
14.
go back to reference Randrianantoandro TN, Kono H, Kubota S. Knowledge and behavior in an animal disease outbreak - evidence from the item count technique in a case of African swine fever in Madagascar. Prev Vet Med. 2015;118(4):483–7.CrossRefPubMed Randrianantoandro TN, Kono H, Kubota S. Knowledge and behavior in an animal disease outbreak - evidence from the item count technique in a case of African swine fever in Madagascar. Prev Vet Med. 2015;118(4):483–7.CrossRefPubMed
15.
go back to reference Gunarathne A, Kubota S, Kumarawadu P, Karunagoda K, Kon H. Is hiding foot and mouth disease sensitive behavior for farmers? A survey study in Sri Lanka. Asian-Australasian J Animal Sci. 2016;29(2):280–7.CrossRef Gunarathne A, Kubota S, Kumarawadu P, Karunagoda K, Kon H. Is hiding foot and mouth disease sensitive behavior for farmers? A survey study in Sri Lanka. Asian-Australasian J Animal Sci. 2016;29(2):280–7.CrossRef
16.
go back to reference Tsuchiya T, Hirai Y, Ono S. A study of the properties of the item count technique. Public Opinion Q. 2007;71(2):253–72.CrossRef Tsuchiya T, Hirai Y, Ono S. A study of the properties of the item count technique. Public Opinion Q. 2007;71(2):253–72.CrossRef
17.
go back to reference Rosenfeld B, Imai K, Shapiro JN. An empirical validation study of popular survey methodologies for sensitive questions. Am J Pol Sci. 2016;60(3):783–802.CrossRef Rosenfeld B, Imai K, Shapiro JN. An empirical validation study of popular survey methodologies for sensitive questions. Am J Pol Sci. 2016;60(3):783–802.CrossRef
18.
go back to reference Tanser F, Hosegood V, Barnighausen T, Herbst K, Nyirenda M, Muhwava W, Newell C, Viljoen J, Mutevedzi T, Newell ML. Cohort profile: Africa Centre demographic information system (ACDIS) and population-based HIV survey. Int J Epidemiol. 2008;37(5):956–62.CrossRefPubMed Tanser F, Hosegood V, Barnighausen T, Herbst K, Nyirenda M, Muhwava W, Newell C, Viljoen J, Mutevedzi T, Newell ML. Cohort profile: Africa Centre demographic information system (ACDIS) and population-based HIV survey. Int J Epidemiol. 2008;37(5):956–62.CrossRefPubMed
19.
go back to reference Zaidi J, Grapsa E, Tanser F, Newell ML, Barnighausen T: Dramatic increase in HIV prevalence after scale-up of antiretroviral treatment. AIDS (London, England) 2013, 27 (14):2301–2305. Zaidi J, Grapsa E, Tanser F, Newell ML, Barnighausen T: Dramatic increase in HIV prevalence after scale-up of antiretroviral treatment. AIDS (London, England) 2013, 27 (14):2301–2305.
20.
go back to reference Tsuchiya T, Hirai Y. Elaborate item count questioning: why do people underreport in item count responses? Survey Res Methods. 2010;4(3):139–49. Tsuchiya T, Hirai Y. Elaborate item count questioning: why do people underreport in item count responses? Survey Res Methods. 2010;4(3):139–49.
21.
go back to reference Nepusz T, Petróczi A, Naughton DP, Epton T, Norman P. Estimating the prevalence of socially sensitive behaviors: attributing guilty and innocent noncompliance with the single sample count method. Psychol Methods. 2014;19(3):334–55.CrossRefPubMed Nepusz T, Petróczi A, Naughton DP, Epton T, Norman P. Estimating the prevalence of socially sensitive behaviors: attributing guilty and innocent noncompliance with the single sample count method. Psychol Methods. 2014;19(3):334–55.CrossRefPubMed
22.
go back to reference Corstange D. Sensitive questions, truthful answers? Modeling the list experiment with LISTIT. Pol Analys. 2009;17(01):45–63.CrossRef Corstange D. Sensitive questions, truthful answers? Modeling the list experiment with LISTIT. Pol Analys. 2009;17(01):45–63.CrossRef
23.
go back to reference Blair G, Imai K: List: statistical methods for the item count technique and list experiment. 8.4 edn; 2010. Blair G, Imai K: List: statistical methods for the item count technique and list experiment. 8.4 edn; 2010.
24.
go back to reference Ulrich R, Schroter H, Striegel H, Simon P. Asking sensitive questions: a statistical power analysis of randomized response models. Psychol Methods. 2012;17(4):623–41.CrossRefPubMed Ulrich R, Schroter H, Striegel H, Simon P. Asking sensitive questions: a statistical power analysis of randomized response models. Psychol Methods. 2012;17(4):623–41.CrossRefPubMed
25.
go back to reference Napper LE, Fisher DG, Reynolds GL, Johnson ME. HIV risk behavior self-report reliability at different recall periods. AIDS Behav. 2010;14(1):152–61.CrossRefPubMed Napper LE, Fisher DG, Reynolds GL, Johnson ME. HIV risk behavior self-report reliability at different recall periods. AIDS Behav. 2010;14(1):152–61.CrossRefPubMed
26.
go back to reference Karlan D, Zinman J. Lying about borrowing. J Eur Econ Assoc. 2008;6(2–3):510–21.CrossRef Karlan D, Zinman J. Lying about borrowing. J Eur Econ Assoc. 2008;6(2–3):510–21.CrossRef
27.
go back to reference Willis G. Cognitive interviewing: a tool for improving questionnaire design. Los Angeles: SAGE Publications; 2004. Willis G. Cognitive interviewing: a tool for improving questionnaire design. Los Angeles: SAGE Publications; 2004.
28.
go back to reference Tourangeau R, Rasinski KA. Cognitive processes underlying context effects in attitude measurement. Psychol Bull. 1988;103(3):299–314.CrossRef Tourangeau R, Rasinski KA. Cognitive processes underlying context effects in attitude measurement. Psychol Bull. 1988;103(3):299–314.CrossRef
Metadata
Title
List randomization for eliciting HIV status and sexual behaviors in rural KwaZulu-Natal, South Africa: a randomized experiment using known true values for validation
Authors
Noah Haber
Guy Harling
Jessica Cohen
Tinofa Mutevedzi
Frank Tanser
Dickman Gareta
Kobus Herbst
Deenan Pillay
Till Bärnighausen
Günther Fink
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2018
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-018-0507-9

Other articles of this Issue 1/2018

BMC Medical Research Methodology 1/2018 Go to the issue