Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Research article

Unweighted regression models perform better than weighted regression techniques for respondent-driven sampling data: results from a simulation study

Authors: Lisa Avery, Nooshin Rotondi, Constance McKnight, Michelle Firestone, Janet Smylie, Michael Rotondi

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

It is unclear whether weighted or unweighted regression is preferred in the analysis of data derived from respondent driven sampling. Our objective was to evaluate the validity of various regression models, with and without weights and with various controls for clustering in the estimation of the risk of group membership from data collected using respondent-driven sampling (RDS).

Methods

Twelve networked populations, with varying levels of homophily and prevalence, based on a known distribution of a continuous predictor were simulated using 1000 RDS samples from each population. Weighted and unweighted binomial and Poisson general linear models, with and without various clustering controls and standard error adjustments were modelled for each sample and evaluated with respect to validity, bias and coverage rate. Population prevalence was also estimated.

Results

In the regression analysis, the unweighted log-link (Poisson) models maintained the nominal type-I error rate across all populations. Bias was substantial and type-I error rates unacceptably high for weighted binomial regression. Coverage rates for the estimation of prevalence were highest using RDS-weighted logistic regression, except at low prevalence (10%) where unweighted models are recommended.

Conclusions

Caution is warranted when undertaking regression analysis of RDS data. Even when reported degree is accurate, low reported degree can unduly influence regression estimates. Unweighted Poisson regression is therefore recommended.
Appendix
Available only for authorised users
Literature
1.
go back to reference Heckathorn DD. Respondent-driven sampling: a new approach to the study of hidden populations. Soc Probl. 1997;44:174–99.CrossRef Heckathorn DD. Respondent-driven sampling: a new approach to the study of hidden populations. Soc Probl. 1997;44:174–99.CrossRef
3.
go back to reference Card KG, Lachowsky NJ, Cui Z, et al. Exploring the role of sex-seeking apps and websites in the social and sexual lives of gay, bisexual and other men who have sex with men: a cross-sectional study. Sex Health. 2017;14:229–37.CrossRef Card KG, Lachowsky NJ, Cui Z, et al. Exploring the role of sex-seeking apps and websites in the social and sexual lives of gay, bisexual and other men who have sex with men: a cross-sectional study. Sex Health. 2017;14:229–37.CrossRef
5.
go back to reference Gile KJ, Johnston LG, Salganik MJ. Diagnostics for respondent driven sampling. J R Stat Soc Ser A: Stat Soc. 2015;178:241–69.CrossRef Gile KJ, Johnston LG, Salganik MJ. Diagnostics for respondent driven sampling. J R Stat Soc Ser A: Stat Soc. 2015;178:241–69.CrossRef
6.
go back to reference White RG, Hakim AJ, Salganik MJ, et al. Strengthening the reporting of observational studies in epidemiology for respondent-driven sampling studies: ‘STROBE-RDS’ statement. J Clin Epidemiol. 2015;68:1463–71.CrossRef White RG, Hakim AJ, Salganik MJ, et al. Strengthening the reporting of observational studies in epidemiology for respondent-driven sampling studies: ‘STROBE-RDS’ statement. J Clin Epidemiol. 2015;68:1463–71.CrossRef
7.
go back to reference Heckathorn DD. Respondent-driven sampling II: deriving valid population estimates from chain-referral samples of hidden populations. Soc Probl. 2002;49:11–34.CrossRef Heckathorn DD. Respondent-driven sampling II: deriving valid population estimates from chain-referral samples of hidden populations. Soc Probl. 2002;49:11–34.CrossRef
8.
go back to reference Rocha LE, Thorson AE, Lambiotte R, et al. Respondent-driven sampling bias induced by community structure and response rates in social networks. J R Stat Soc Ser A: Stat Soc. 2017;180:99–118.CrossRef Rocha LE, Thorson AE, Lambiotte R, et al. Respondent-driven sampling bias induced by community structure and response rates in social networks. J R Stat Soc Ser A: Stat Soc. 2017;180:99–118.CrossRef
9.
go back to reference Spiller MW, Gile KJ, Handcock MS, et al. Evaluating variance estimators for respondent-driven sampling. J Surv Stat Methodol. 2018;6:23–45.CrossRef Spiller MW, Gile KJ, Handcock MS, et al. Evaluating variance estimators for respondent-driven sampling. J Surv Stat Methodol. 2018;6:23–45.CrossRef
10.
go back to reference Baraff AJ, McCormick TH, Raftery AE. Estimating uncertainty in respondent-driven sampling using a tree bootstrap method. Proc Natl Acad Sci U S A. 2016;113:14668–73.CrossRef Baraff AJ, McCormick TH, Raftery AE. Estimating uncertainty in respondent-driven sampling using a tree bootstrap method. Proc Natl Acad Sci U S A. 2016;113:14668–73.CrossRef
11.
go back to reference McCreesh N, Frost SDW, Seeley J, et al. Evaluation of respondent-driven sampling. Epidemiology. 2012;23:138–47.CrossRef McCreesh N, Frost SDW, Seeley J, et al. Evaluation of respondent-driven sampling. Epidemiology. 2012;23:138–47.CrossRef
13.
go back to reference Schwartz S, Papworth E, Thiam-Niangoin M, et al. An urgent need for integration of family planning services into HIV care. J Acquir Immune Defic Syndr. 2015;68:S91–8.CrossRef Schwartz S, Papworth E, Thiam-Niangoin M, et al. An urgent need for integration of family planning services into HIV care. J Acquir Immune Defic Syndr. 2015;68:S91–8.CrossRef
14.
go back to reference de Matos MA, da Silva França DD, dos Santos Carneiro MA, et al. Viral hepatitis in female sex workers using the respondent-driven sampling. Rev Saude Publica. 2017;51:1–11.CrossRef de Matos MA, da Silva França DD, dos Santos Carneiro MA, et al. Viral hepatitis in female sex workers using the respondent-driven sampling. Rev Saude Publica. 2017;51:1–11.CrossRef
15.
go back to reference Scheim AI, Zong X, Giblon R, et al. Disparities in access to family physicians among transgender people in Ontario, Canada. Int J Transgend. 2017;18:343–52.CrossRef Scheim AI, Zong X, Giblon R, et al. Disparities in access to family physicians among transgender people in Ontario, Canada. Int J Transgend. 2017;18:343–52.CrossRef
16.
go back to reference Pan X, Wu M, Ma Q, et al. High prevalence of HIV among men who have sex with men in Zhejiang, China: a respondent-driven sampling survey. BMJ Open. 2015;5:1–7.CrossRef Pan X, Wu M, Ma Q, et al. High prevalence of HIV among men who have sex with men in Zhejiang, China: a respondent-driven sampling survey. BMJ Open. 2015;5:1–7.CrossRef
17.
go back to reference Hatzakis A, Sypsa V, Paraskevis D, et al. Design and baseline findings of a large-scale rapid response to an HIV outbreak in people who inject drugs in Athens, Greece: the ARISTOTLE programme. Addiction. 2015;110:1453–67.CrossRef Hatzakis A, Sypsa V, Paraskevis D, et al. Design and baseline findings of a large-scale rapid response to an HIV outbreak in people who inject drugs in Athens, Greece: the ARISTOTLE programme. Addiction. 2015;110:1453–67.CrossRef
18.
go back to reference Maragh-Bass AC, Powell C, Park J, et al. Sociodemographic and access-related correlates of health-care utilization among African American injection drug users: the BESURE study. J Ethn Subst Abus. 2017;16:344–62.CrossRef Maragh-Bass AC, Powell C, Park J, et al. Sociodemographic and access-related correlates of health-care utilization among African American injection drug users: the BESURE study. J Ethn Subst Abus. 2017;16:344–62.CrossRef
20.
go back to reference Spiller MW, Cameron C, Heckathorn DD. Respondent-driven sampling analysis tool (RDSAT) version 7.1 copyright. Cornell University; 2012. Spiller MW, Cameron C, Heckathorn DD. Respondent-driven sampling analysis tool (RDSAT) version 7.1 copyright. Cornell University; 2012.
22.
go back to reference Beckett M, Firestone MA, McKnight CD, et al. A cross-sectional analysis of the relationship between diabetes and health access barriers in an urban first nations population in Canada. BMJ Open. 2018;8:e018272.CrossRef Beckett M, Firestone MA, McKnight CD, et al. A cross-sectional analysis of the relationship between diabetes and health access barriers in an urban first nations population in Canada. BMJ Open. 2018;8:e018272.CrossRef
24.
go back to reference Hubbard AE, Ahern J, Fleischer NL, et al. To GEE or not to GEE. Epidemiology. 2010;21:467–74.CrossRef Hubbard AE, Ahern J, Fleischer NL, et al. To GEE or not to GEE. Epidemiology. 2010;21:467–74.CrossRef
25.
go back to reference Rao S, LaRocque R, Jentes E, et al. Comparison of methods for clustered data analysis in a non-ideal situation: results from an evaluation of predictors of yellow fever vaccine refusal in the global TravEpiNet (GTEN) consortium. Int J Stat Med Res. 2014;3:215–23.CrossRef Rao S, LaRocque R, Jentes E, et al. Comparison of methods for clustered data analysis in a non-ideal situation: results from an evaluation of predictors of yellow fever vaccine refusal in the global TravEpiNet (GTEN) consortium. Int J Stat Med Res. 2014;3:215–23.CrossRef
29.
go back to reference Venables W, Ripley B. Modern Applied Statistics with S. Fourth Edition. New York; Springer. 2002. Venables W, Ripley B. Modern Applied Statistics with S. Fourth Edition. New York; Springer. 2002.
31.
go back to reference Volz E, Heckathorn DD. Probability based estimation theory for respondent driven sampling. J Off Stat. 2008;24:79–97. Volz E, Heckathorn DD. Probability based estimation theory for respondent driven sampling. J Off Stat. 2008;24:79–97.
34.
go back to reference Morel G. Logistic regression under complex survey designs. Surv Methodol Stat Can. 1989;15:203–23. Morel G. Logistic regression under complex survey designs. Surv Methodol Stat Can. 1989;15:203–23.
36.
go back to reference Kuhns LM, Hotton AL, Schneider J, et al. Use of pre-exposure prophylaxis (PrEP) in young men who have sex with men is associated with race, sexual risk behavior and peer network size. AIDS Behav. 2017;21:1376–82.CrossRef Kuhns LM, Hotton AL, Schneider J, et al. Use of pre-exposure prophylaxis (PrEP) in young men who have sex with men is associated with race, sexual risk behavior and peer network size. AIDS Behav. 2017;21:1376–82.CrossRef
37.
go back to reference Li R, Wang H, Pan X, et al. Prevalence of condomless anal intercourse and recent HIV testing and their associated factors among men who have sex with men in Hangzhou, China: a respondent-driven sampling survey. PLoS One. 2017;12:1–18. Li R, Wang H, Pan X, et al. Prevalence of condomless anal intercourse and recent HIV testing and their associated factors among men who have sex with men in Hangzhou, China: a respondent-driven sampling survey. PLoS One. 2017;12:1–18.
38.
go back to reference Pando MA, Dolezal C, Marone RO, et al. High acceptability of rapid HIV self-testing among a diverse sample of MSM from Buenos Aires, Argentina. PLoS One. 2017;12:1–12.CrossRef Pando MA, Dolezal C, Marone RO, et al. High acceptability of rapid HIV self-testing among a diverse sample of MSM from Buenos Aires, Argentina. PLoS One. 2017;12:1–12.CrossRef
40.
go back to reference Mmbaga EJ, Moen K, Makyao N, et al. HIV and STI s among men who have sex with men in Dodoma municipality, Tanzania: a cross-sectional study. Sex Transm Infect. 2017;93:314–9.CrossRef Mmbaga EJ, Moen K, Makyao N, et al. HIV and STI s among men who have sex with men in Dodoma municipality, Tanzania: a cross-sectional study. Sex Transm Infect. 2017;93:314–9.CrossRef
41.
go back to reference Donner A, Klar N. Design and analysis of cluster randomization trials in health research. New York: Oxford University Press; 2010. Donner A, Klar N. Design and analysis of cluster randomization trials in health research. New York: Oxford University Press; 2010.
42.
go back to reference Goel S, Salganik MJ. Assessing respondent-driven sampling. Proc Natl Acad Sci U S A. 2010;107:6743–7.CrossRef Goel S, Salganik MJ. Assessing respondent-driven sampling. Proc Natl Acad Sci U S A. 2010;107:6743–7.CrossRef
43.
go back to reference Lohr SL, Liu J. A comparison of weighted and unweighted analyses in the national crime victimization survey. J Quant Criminol. 1994;10:343–60.CrossRef Lohr SL, Liu J. A comparison of weighted and unweighted analyses in the national crime victimization survey. J Quant Criminol. 1994;10:343–60.CrossRef
44.
go back to reference Miratrix LW, Sekhon JS, Theodoridis AG, et al. Worth weighting? How to think about and use weights in survey experiments. arXiv. 2017;1703(06808):1–49. Miratrix LW, Sekhon JS, Theodoridis AG, et al. Worth weighting? How to think about and use weights in survey experiments. arXiv. 2017;1703(06808):1–49.
45.
go back to reference Reed E, Erausquin JT, Biradavolu M, et al. Non-barrier contraceptive use and relation to condom use behaviour by partner type among female sex workers in Andhra Pradesh, India. J Fam Plann Reprod Health Care. 2017;43:60–6.CrossRef Reed E, Erausquin JT, Biradavolu M, et al. Non-barrier contraceptive use and relation to condom use behaviour by partner type among female sex workers in Andhra Pradesh, India. J Fam Plann Reprod Health Care. 2017;43:60–6.CrossRef
Metadata
Title
Unweighted regression models perform better than weighted regression techniques for respondent-driven sampling data: results from a simulation study
Authors
Lisa Avery
Nooshin Rotondi
Constance McKnight
Michelle Firestone
Janet Smylie
Michael Rotondi
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0842-5

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue