Skip to main content
Top
Published in: Prevention Science 4/2010

01-12-2010

Handling Missing Data in Randomized Experiments with Noncompliance

Authors: Booil Jo, Elizabeth M. Ginexi, Nicholas S. Ialongo

Published in: Prevention Science | Issue 4/2010

Login to get access

Abstract

Treatment noncompliance and missing outcomes at posttreatment assessments are common problems in field experiments in naturalistic settings. Although the two complications often occur simultaneously, statistical methods that address both complications have not been routinely considered in data analysis practice in the prevention research field. This paper shows that identification and estimation of causal treatment effects considering both noncompliance and missing outcomes can be relatively easily conducted under various missing data assumptions. We review a few assumptions on missing data in the presence of noncompliance, including the latent ignorability proposed by Frangakis and Rubin (Biometrika 86:365–379, 1999), and show how these assumptions can be used in the parametric complier average causal effect (CACE) estimation framework. As an easy way of sensitivity analysis, we propose the use of alternative missing data assumptions, which will provide a range of causal effect estimates. In this way, we are less likely to settle with a possibly biased causal effect estimate based on a single assumption. We demonstrate how alternative missing data assumptions affect identification of causal effects, focusing on the CACE. The data from the Johns Hopkins School Intervention Study (Ialongo et al., Am J Community Psychol 27:599–642, 1999) will be used as an example.
Appendix
Available only for authorised users
Literature
go back to reference Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–455.CrossRef Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–455.CrossRef
go back to reference Bloom, H. S. (1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8, 225–246.CrossRef Bloom, H. S. (1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8, 225–246.CrossRef
go back to reference Dunn, G., Maracy, M., Dowrick, C., Ayuso-Mateos, J. L., Dalgard, O. S., Page, H., et al. (2003). Estimating psychological treatment effects from a randomized controlled trial with both non-compliance and loss to follow-up. British Journal of Psychiatry, 183, 323–331.CrossRefPubMed Dunn, G., Maracy, M., Dowrick, C., Ayuso-Mateos, J. L., Dalgard, O. S., Page, H., et al. (2003). Estimating psychological treatment effects from a randomized controlled trial with both non-compliance and loss to follow-up. British Journal of Psychiatry, 183, 323–331.CrossRefPubMed
go back to reference Emsley, R., Dunn, G., & White, I. R. (2010). Mediation and moderation of treatment effects in randomised controlled trials of complex interventions. Statistical Methods in Medical Research. doi:10.1177/0962280209105014. Emsley, R., Dunn, G., & White, I. R. (2010). Mediation and moderation of treatment effects in randomised controlled trials of complex interventions. Statistical Methods in Medical Research. doi:10.​1177/​0962280209105014​.
go back to reference Frangakis, C. E. & Rubin, D. B. (1999). Addressing complications of intention-to-treat analysis in the presence of all-or-none treatment-noncompliance and subsequent missing outcomes. Biometrika, 86, 365–379.CrossRef Frangakis, C. E. & Rubin, D. B. (1999). Addressing complications of intention-to-treat analysis in the presence of all-or-none treatment-noncompliance and subsequent missing outcomes. Biometrika, 86, 365–379.CrossRef
go back to reference Frangakis, C. E. & Rubin, D. B. (2002) Principal stratification in causal inference. Biometrics, 58, 21–29.CrossRefPubMed Frangakis, C. E. & Rubin, D. B. (2002) Principal stratification in causal inference. Biometrics, 58, 21–29.CrossRefPubMed
go back to reference Frangakis, C. E., Rubin, D. B., & Zhou, X. H. (2002). Clustered encouragement design with individual noncompliance: Bayesian inference and application to advance directive forms. Biostatistics, 3, 147–164.CrossRefPubMed Frangakis, C. E., Rubin, D. B., & Zhou, X. H. (2002). Clustered encouragement design with individual noncompliance: Bayesian inference and application to advance directive forms. Biostatistics, 3, 147–164.CrossRefPubMed
go back to reference Hirano, K., Imbens, G. W., Rubin, D. B., & Zhou, X. H. (2000). Assessing the effect of an influenza vaccine in an encouragement design. Biostatistics, 1, 69–88.CrossRefPubMed Hirano, K., Imbens, G. W., Rubin, D. B., & Zhou, X. H. (2000). Assessing the effect of an influenza vaccine in an encouragement design. Biostatistics, 1, 69–88.CrossRefPubMed
go back to reference Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945–960.CrossRef Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945–960.CrossRef
go back to reference Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression and antisocial behavior. American Journal of Community Psychology, 27, 599–642.CrossRefPubMed Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression and antisocial behavior. American Journal of Community Psychology, 27, 599–642.CrossRefPubMed
go back to reference Imbens, G. W. & Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with non-compliance. Annals of Statistics, 25, 305–327.CrossRef Imbens, G. W. & Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with non-compliance. Annals of Statistics, 25, 305–327.CrossRef
go back to reference Jo, B. (2002a). Statistical power in randomized intervention studies with noncompliance. Psychological Methods, 7, 178–193.CrossRefPubMed Jo, B. (2002a). Statistical power in randomized intervention studies with noncompliance. Psychological Methods, 7, 178–193.CrossRefPubMed
go back to reference Jo, B. (2002b). Estimating intervention effects with noncompliance: Alternative model specifications. Journal of Educational and Behavioral Statistics, 27, 385–420.CrossRef Jo, B. (2002b). Estimating intervention effects with noncompliance: Alternative model specifications. Journal of Educational and Behavioral Statistics, 27, 385–420.CrossRef
go back to reference Jo, B. (2008a). Bias mechanisms in intention-to-treat analysis with data subject to treatment noncompliance and missing outcomes. Journal of Educational and Behavioral Statistics, 33, 158–185.CrossRef Jo, B. (2008a). Bias mechanisms in intention-to-treat analysis with data subject to treatment noncompliance and missing outcomes. Journal of Educational and Behavioral Statistics, 33, 158–185.CrossRef
go back to reference Jo, B. (2008b). Causal inference in randomized experiments with mediational processes. Psychological Methods, 13, 314–336.CrossRefPubMed Jo, B. (2008b). Causal inference in randomized experiments with mediational processes. Psychological Methods, 13, 314–336.CrossRefPubMed
go back to reference Jo, B., Asparouhov, T., Muthén, B. O., Ialongo, N. S., & Brown, C. H. (2008). Cluster randomized trials with treatment noncompliance. Psychological Methods, 13, 1–18.CrossRefPubMed Jo, B., Asparouhov, T., Muthén, B. O., Ialongo, N. S., & Brown, C. H. (2008). Cluster randomized trials with treatment noncompliance. Psychological Methods, 13, 1–18.CrossRefPubMed
go back to reference Jo, B., & Vinokur, A. (2010). Sensitivity analysis and bounding of causal effects with alternative identifying assumptions. Journal of Educational and Behavioral Statistics, in press. Jo, B., & Vinokur, A. (2010). Sensitivity analysis and bounding of causal effects with alternative identifying assumptions. Journal of Educational and Behavioral Statistics, in press.
go back to reference Kellam, S. G., Branch, J. D., Agrawal, K. C., & Ensminger, M. E. (1975). Mental health and going to school: The Woodlawn program of assessment, early intervention, and evaluation. Chicago: University of Chicago Press. Kellam, S. G., Branch, J. D., Agrawal, K. C., & Ensminger, M. E. (1975). Mental health and going to school: The Woodlawn program of assessment, early intervention, and evaluation. Chicago: University of Chicago Press.
go back to reference Little, R. J. A. & Rubin, D. B. (2002). Statistical analysis with missing data. New York: Wiley. Little, R. J. A. & Rubin, D. B. (2002). Statistical analysis with missing data. New York: Wiley.
go back to reference Little, R. J. A., & Yau, L. (1998). Statistical techniques for analyzing data from prevention trials: Treatment of no-shows using Rubin’s causal model. Psychological Methods, 3, 147–159.CrossRef Little, R. J. A., & Yau, L. (1998). Statistical techniques for analyzing data from prevention trials: Treatment of no-shows using Rubin’s causal model. Psychological Methods, 3, 147–159.CrossRef
go back to reference Mattei, A., & Mealli, F. (2007). Application of the principal stratification approach to the Faenza randomized experiment on breast self-examination. Biometrics, 63, 437–446.CrossRefPubMed Mattei, A., & Mealli, F. (2007). Application of the principal stratification approach to the Faenza randomized experiment on breast self-examination. Biometrics, 63, 437–446.CrossRefPubMed
go back to reference Mealli, F., Imbens, G. W., Ferro, S., & Biggeri A. (2004). Analyzing a randomized trial on breast self-examination with noncompliance and missing outcomes. Biostatistics, 5, 207–222.CrossRefPubMed Mealli, F., Imbens, G. W., Ferro, S., & Biggeri A. (2004). Analyzing a randomized trial on breast self-examination with noncompliance and missing outcomes. Biostatistics, 5, 207–222.CrossRefPubMed
go back to reference Muthén, L. K., & Muthén, B. O. (1998–2009). Mplus user’s guide. Los Angeles: Muthén & Muthén. Muthén, L. K., & Muthén, B. O. (1998–2009). Mplus user’s guide. Los Angeles: Muthén & Muthén.
go back to reference Neyman, J. (1923). On the application of probability theory to agricultural experiments. Section 9 translated in Statistical Science, 5, 465–480 (1990). Neyman, J. (1923). On the application of probability theory to agricultural experiments. Section 9 translated in Statistical Science, 5, 465–480 (1990).
go back to reference O’Malley, A. J., & Normand, S. L. T. (2004). Likelihood methods for treatment noncompliance and subsequent nonresponse in randomized trials. Biometrics, 61, 325–334.CrossRef O’Malley, A. J., & Normand, S. L. T. (2004). Likelihood methods for treatment noncompliance and subsequent nonresponse in randomized trials. Biometrics, 61, 325–334.CrossRef
go back to reference Peng, Y., Little, R. J., & Raghunathan, T. E. (2004). An extended general location model for causal inferences from data subject to noncompliance and missing values. Biometrics, 60, 598–607.CrossRefPubMed Peng, Y., Little, R. J., & Raghunathan, T. E. (2004). An extended general location model for causal inferences from data subject to noncompliance and missing values. Biometrics, 60, 598–607.CrossRefPubMed
go back to reference Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6, 34–58.CrossRef Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6, 34–58.CrossRef
go back to reference Rubin, D. B. (1980). Discussion of “randomization analysis of experimental data in the Fisher randomization test” by D. Basu. Journal of the American Statistical Association, 75, 591–593.CrossRef Rubin, D. B. (1980). Discussion of “randomization analysis of experimental data in the Fisher randomization test” by D. Basu. Journal of the American Statistical Association, 75, 591–593.CrossRef
go back to reference Rubin, D. B. (1990). Comment on “Neyman (1923) and causal inference in experiments and observational studies.” Statistical Science, 5, 472–480. Rubin, D. B. (1990). Comment on “Neyman (1923) and causal inference in experiments and observational studies.” Statistical Science, 5, 472–480.
go back to reference Schafer, J. L. (1997). Analysis of incomplete multivariate data. London: CRC.CrossRef Schafer, J. L. (1997). Analysis of incomplete multivariate data. London: CRC.CrossRef
go back to reference Sobel, M. E. (2006).What do randomized studies of housing mobility demonstrate: Causal inference in the face of interference. Journal of the American Statistical Association, 101, 1398–1407.CrossRef Sobel, M. E. (2006).What do randomized studies of housing mobility demonstrate: Causal inference in the face of interference. Journal of the American Statistical Association, 101, 1398–1407.CrossRef
go back to reference Stuart, E. A., Perry, D. F., Le, H-N, & Ialongo, N. S. (2008). Estimating intervention effects of prevention programs: Accounting for noncompliance. Prevention Science, 9, 288–298.CrossRefPubMed Stuart, E. A., Perry, D. F., Le, H-N, & Ialongo, N. S. (2008). Estimating intervention effects of prevention programs: Accounting for noncompliance. Prevention Science, 9, 288–298.CrossRefPubMed
go back to reference Werthamer-Larsson, L., Kellam, S. G., & Wheeler, L. (1991). Effect of first-grade classroom environment on child shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology, 19, 585–602.CrossRefPubMed Werthamer-Larsson, L., Kellam, S. G., & Wheeler, L. (1991). Effect of first-grade classroom environment on child shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology, 19, 585–602.CrossRefPubMed
Metadata
Title
Handling Missing Data in Randomized Experiments with Noncompliance
Authors
Booil Jo
Elizabeth M. Ginexi
Nicholas S. Ialongo
Publication date
01-12-2010
Publisher
Springer US
Published in
Prevention Science / Issue 4/2010
Print ISSN: 1389-4986
Electronic ISSN: 1573-6695
DOI
https://doi.org/10.1007/s11121-010-0175-4

Other articles of this Issue 4/2010

Prevention Science 4/2010 Go to the issue