Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2013

Open Access 01-12-2013 | Research article

Comparison of population-averaged and cluster-specific models for the analysis of cluster randomized trials with missing binary outcomes: a simulation study

Authors: Jinhui Ma, Parminder Raina, Joseph Beyene, Lehana Thabane

Published in: BMC Medical Research Methodology | Issue 1/2013

Login to get access

Abstracts

Background

The objective of this simulation study is to compare the accuracy and efficiency of population-averaged (i.e. generalized estimating equations (GEE)) and cluster-specific (i.e. random-effects logistic regression (RELR)) models for analyzing data from cluster randomized trials (CRTs) with missing binary responses.

Methods

In this simulation study, clustered responses were generated from a beta-binomial distribution. The number of clusters per trial arm, the number of subjects per cluster, intra-cluster correlation coefficient, and the percentage of missing data were allowed to vary. Under the assumption of covariate dependent missingness, missing outcomes were handled by complete case analysis, standard multiple imputation (MI) and within-cluster MI strategies. Data were analyzed using GEE and RELR. Performance of the methods was assessed using standardized bias, empirical standard error, root mean squared error (RMSE), and coverage probability.

Results

GEE performs well on all four measures — provided the downward bias of the standard error (when the number of clusters per arm is small) is adjusted appropriately — under the following scenarios: complete case analysis for CRTs with a small amount of missing data; standard MI for CRTs with variance inflation factor (VIF) <3; within-cluster MI for CRTs with VIF≥3 and cluster size>50. RELR performs well only when a small amount of data was missing, and complete case analysis was applied.

Conclusion

GEE performs well as long as appropriate missing data strategies are adopted based on the design of CRTs and the percentage of missing data. In contrast, RELR does not perform well when either standard or within-cluster MI strategy is applied prior to the analysis.
Appendix
Available only for authorised users
Literature
1.
go back to reference Donner A, Klar N: Design and Analysis of Cluster Randomization Trials in Health Research. 2000, New York: John Wiley & Sons Donner A, Klar N: Design and Analysis of Cluster Randomization Trials in Health Research. 2000, New York: John Wiley & Sons
2.
go back to reference Donner A, Brown KS, Brasher P: A methodological review of non-therapeutic intervention trials employing cluster randomization, 1979–1989. Int J Epidemiol. 1990, 19 (4): 795-800. 10.1093/ije/19.4.795.CrossRefPubMed Donner A, Brown KS, Brasher P: A methodological review of non-therapeutic intervention trials employing cluster randomization, 1979–1989. Int J Epidemiol. 1990, 19 (4): 795-800. 10.1093/ije/19.4.795.CrossRefPubMed
3.
go back to reference Little RJA, Rubin DB: Statistical analysis with missing data. 2002, New-York: John Wiley & Sons, 2 Little RJA, Rubin DB: Statistical analysis with missing data. 2002, New-York: John Wiley & Sons, 2
4.
go back to reference Groenwold RH, Donders AR, Roes KC, Harrell FE, Moons KG: Dealing with missing outcome data in randomized trials and observational studies. Am J Epidemiol. 2012, 175 (3): 210-217. 10.1093/aje/kwr302.CrossRefPubMed Groenwold RH, Donders AR, Roes KC, Harrell FE, Moons KG: Dealing with missing outcome data in randomized trials and observational studies. Am J Epidemiol. 2012, 175 (3): 210-217. 10.1093/aje/kwr302.CrossRefPubMed
5.
go back to reference Liublinska V, Rubin DB: Re: "dealing with missing outcome data in randomized trials and observational studies. Am J Epidemiol. 2012, 176 (4): 357-358. 10.1093/aje/kws215.CrossRefPubMed Liublinska V, Rubin DB: Re: "dealing with missing outcome data in randomized trials and observational studies. Am J Epidemiol. 2012, 176 (4): 357-358. 10.1093/aje/kws215.CrossRefPubMed
6.
go back to reference Ma J, Raina P, Beyene J, Thabane L: Comparing the performance of different multiple imputation strategies for missing binary outcomes in cluster randomized trials: a simulation study. J Open Access Med Stat. 2012, 2: 93-103. Ma J, Raina P, Beyene J, Thabane L: Comparing the performance of different multiple imputation strategies for missing binary outcomes in cluster randomized trials: a simulation study. J Open Access Med Stat. 2012, 2: 93-103.
7.
go back to reference Zeger SL, Liang KY, Albert PS: Models for longitudinal data: a generalized estimating equation approach. Biometrics. 1988, 44: 1049-1060. 10.2307/2531734.CrossRefPubMed Zeger SL, Liang KY, Albert PS: Models for longitudinal data: a generalized estimating equation approach. Biometrics. 1988, 44: 1049-1060. 10.2307/2531734.CrossRefPubMed
8.
go back to reference Austin PC: A comparison of the statistical power of different methods for the analysis of cluster randomization trials with binary outcomes. Stat Med. 2007, 26: 3550-3565. 10.1002/sim.2813.CrossRefPubMed Austin PC: A comparison of the statistical power of different methods for the analysis of cluster randomization trials with binary outcomes. Stat Med. 2007, 26: 3550-3565. 10.1002/sim.2813.CrossRefPubMed
9.
go back to reference Bellamy SL, Gibbard R, Hancock L, Howley P, Kennedy B, Klar N, Lipsitz S, Ryan L: Analysis of dichotomous outcome data for community intervention studies. Stat Methods Med Res. 2000, 9: 135-159. 10.1191/096228000672549488.CrossRefPubMed Bellamy SL, Gibbard R, Hancock L, Howley P, Kennedy B, Klar N, Lipsitz S, Ryan L: Analysis of dichotomous outcome data for community intervention studies. Stat Methods Med Res. 2000, 9: 135-159. 10.1191/096228000672549488.CrossRefPubMed
10.
go back to reference Ukoumunne OC, Carlin JB, Gulliford MC: A simulation study of odds ratio estimation for binary outcomes from cluster randomized trials. Stat Med. 2007, 26 (18): 3415-3428.CrossRefPubMed Ukoumunne OC, Carlin JB, Gulliford MC: A simulation study of odds ratio estimation for binary outcomes from cluster randomized trials. Stat Med. 2007, 26 (18): 3415-3428.CrossRefPubMed
11.
go back to reference Ma J, Akhtar-Danesh N, Dolovich L, Thabane L: CHAT investigators. Imputation strategies for missing binary outcomes in cluster randomized trials. BMC Med Res Methodol. 2011, 11: 18-10.1186/1471-2288-11-18.CrossRefPubMedPubMedCentral Ma J, Akhtar-Danesh N, Dolovich L, Thabane L: CHAT investigators. Imputation strategies for missing binary outcomes in cluster randomized trials. BMC Med Res Methodol. 2011, 11: 18-10.1186/1471-2288-11-18.CrossRefPubMedPubMedCentral
12.
go back to reference Liang K, Zeger S: Longitudinal data analysis using generalized linear models. Biometrika. 1986, 73 (1): 13-22. 10.1093/biomet/73.1.13.CrossRef Liang K, Zeger S: Longitudinal data analysis using generalized linear models. Biometrika. 1986, 73 (1): 13-22. 10.1093/biomet/73.1.13.CrossRef
13.
go back to reference Murray DM, Varnell SP, Blitstein JL: Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health. 2004, 94: 423-432. 10.2105/AJPH.94.3.423.CrossRefPubMedPubMedCentral Murray DM, Varnell SP, Blitstein JL: Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health. 2004, 94: 423-432. 10.2105/AJPH.94.3.423.CrossRefPubMedPubMedCentral
14.
go back to reference Mancl LA, DeRouen TA: A covariance estimator for GEE with improved small-sample properties. Biometrics. 2001, 57: 126-134. 10.1111/j.0006-341X.2001.00126.x.CrossRefPubMed Mancl LA, DeRouen TA: A covariance estimator for GEE with improved small-sample properties. Biometrics. 2001, 57: 126-134. 10.1111/j.0006-341X.2001.00126.x.CrossRefPubMed
15.
go back to reference McCulloch CE, Searle SR: Generalized, Linear and Mixed Models. 2001, New York: John Wiley & Sons Inc McCulloch CE, Searle SR: Generalized, Linear and Mixed Models. 2001, New York: John Wiley & Sons Inc
16.
go back to reference Feng Z, Diehr P, Peterson A, McLerran D: Selected statistical issues in group randomized trials. Annu Rev Public Health. 2001, 22: 167-187. 10.1146/annurev.publhealth.22.1.167.CrossRefPubMed Feng Z, Diehr P, Peterson A, McLerran D: Selected statistical issues in group randomized trials. Annu Rev Public Health. 2001, 22: 167-187. 10.1146/annurev.publhealth.22.1.167.CrossRefPubMed
17.
go back to reference Neuhaus JM: Statistical methods for longitudinal and clustered designs with binary responses. Stat Methods Med Res. 1992, 1: 249-273. 10.1177/096228029200100303.CrossRefPubMed Neuhaus JM: Statistical methods for longitudinal and clustered designs with binary responses. Stat Methods Med Res. 1992, 1: 249-273. 10.1177/096228029200100303.CrossRefPubMed
18.
go back to reference Rubin DB: Multiple imputation after 18+ years. J Am Stat Assoc. 1996, 91: 473-489. 10.1080/01621459.1996.10476908.CrossRef Rubin DB: Multiple imputation after 18+ years. J Am Stat Assoc. 1996, 91: 473-489. 10.1080/01621459.1996.10476908.CrossRef
19.
go back to reference Barnard J, Rubin DB: Small-sample degrees of freedom with multiple imputation. Biometrika. 1999, 86: 949-955.CrossRef Barnard J, Rubin DB: Small-sample degrees of freedom with multiple imputation. Biometrika. 1999, 86: 949-955.CrossRef
20.
go back to reference Eldridge SM, Ashby D, Feder GS, Rudnicka AR, Ukoumunne OC: Lessons for cluster randomized trials in the twenty-first century: A systematic review of trials in primary care. Clin Trials. 2004, 1 (1): 80-90. 10.1191/1740774504cn006rr.CrossRefPubMed Eldridge SM, Ashby D, Feder GS, Rudnicka AR, Ukoumunne OC: Lessons for cluster randomized trials in the twenty-first century: A systematic review of trials in primary care. Clin Trials. 2004, 1 (1): 80-90. 10.1191/1740774504cn006rr.CrossRefPubMed
21.
go back to reference Donner A: An empirical study of cluster randomization. Int J Epidemiol. 1982, 11 (3): 283-286. 10.1093/ije/11.3.283.CrossRefPubMed Donner A: An empirical study of cluster randomization. Int J Epidemiol. 1982, 11 (3): 283-286. 10.1093/ije/11.3.283.CrossRefPubMed
22.
go back to reference Lee EW, Bubin N: Estimation and sample size considerations for clustered binary responses. Stat Med. 1994, 13: 1241-1252. 10.1002/sim.4780131206.CrossRefPubMed Lee EW, Bubin N: Estimation and sample size considerations for clustered binary responses. Stat Med. 1994, 13: 1241-1252. 10.1002/sim.4780131206.CrossRefPubMed
23.
go back to reference Burton A, Altman DG, Royston P, Holder RL: The design of simulation studies in medical statistics. Stat Med. 2006, 25 (24): 4279-4292. 10.1002/sim.2673.CrossRefPubMed Burton A, Altman DG, Royston P, Holder RL: The design of simulation studies in medical statistics. Stat Med. 2006, 25 (24): 4279-4292. 10.1002/sim.2673.CrossRefPubMed
24.
go back to reference Taljaard M, Donner A, Klar N: Imputation strategies for missing continuous outcomes in cluster randomized trials. Biom J. 2008, 50 (3): 329-345. 10.1002/bimj.200710423.CrossRefPubMed Taljaard M, Donner A, Klar N: Imputation strategies for missing continuous outcomes in cluster randomized trials. Biom J. 2008, 50 (3): 329-345. 10.1002/bimj.200710423.CrossRefPubMed
25.
go back to reference Brown EC, Graham JW, Hawkins JD, Arthur MW, Baldwin MM, Oesterle S, Briney JS, Catalano RF, Abbott RD: Design and analysis of the community youth development study longitudinal cohort sample. Eval Rev. 2009, 33: 311-324. 10.1177/0193841X09337356.CrossRefPubMedPubMedCentral Brown EC, Graham JW, Hawkins JD, Arthur MW, Baldwin MM, Oesterle S, Briney JS, Catalano RF, Abbott RD: Design and analysis of the community youth development study longitudinal cohort sample. Eval Rev. 2009, 33: 311-324. 10.1177/0193841X09337356.CrossRefPubMedPubMedCentral
26.
go back to reference Hawkins JD, Brown EC, Oesterle S, Arthur MW, Abbott RD, Catalano RF: Early effects of communities that care on targeted risks and initiation of delinquent behavior and substance use. J Adolesc Health. 2008, 43: 15-22. 10.1016/j.jadohealth.2008.01.022.CrossRefPubMed Hawkins JD, Brown EC, Oesterle S, Arthur MW, Abbott RD, Catalano RF: Early effects of communities that care on targeted risks and initiation of delinquent behavior and substance use. J Adolesc Health. 2008, 43: 15-22. 10.1016/j.jadohealth.2008.01.022.CrossRefPubMed
27.
go back to reference Hawkins JD, Oesterle S, Brown EC, Arthur MW, Abbott RD, Fagan AA, Catalano RF: Results of a type 2 translational research trial to prevent adolescent drug use and delinquency. Arch Pediatr Adolesc Med. 2009, 163: 789-798. 10.1001/archpediatrics.2009.141.CrossRefPubMedPubMedCentral Hawkins JD, Oesterle S, Brown EC, Arthur MW, Abbott RD, Fagan AA, Catalano RF: Results of a type 2 translational research trial to prevent adolescent drug use and delinquency. Arch Pediatr Adolesc Med. 2009, 163: 789-798. 10.1001/archpediatrics.2009.141.CrossRefPubMedPubMedCentral
28.
go back to reference Clark NM, Shah S, Dodge JA, Thomas LJ, Andridge RR, Awad D, Little RJA: An evaluation of asthma interventions for preteen students. J Sch Health. 2010, 80: 80-87. 10.1111/j.1746-1561.2009.00469.x.CrossRefPubMedPubMedCentral Clark NM, Shah S, Dodge JA, Thomas LJ, Andridge RR, Awad D, Little RJA: An evaluation of asthma interventions for preteen students. J Sch Health. 2010, 80: 80-87. 10.1111/j.1746-1561.2009.00469.x.CrossRefPubMedPubMedCentral
29.
go back to reference French SA, Story M, Fulkerson JA, Himes JH, Hannan P, Neumark-Sztainer D, Ensrud K: Increasing weight-bearing physical activity and calcium-rich foods to promote bone mass gains among 9–11 year old girls: outcomes of the cal-girls study. Int J Behav Nutr Phys Act. 2005, 2: 8-10.1186/1479-5868-2-8.CrossRefPubMedPubMedCentral French SA, Story M, Fulkerson JA, Himes JH, Hannan P, Neumark-Sztainer D, Ensrud K: Increasing weight-bearing physical activity and calcium-rich foods to promote bone mass gains among 9–11 year old girls: outcomes of the cal-girls study. Int J Behav Nutr Phys Act. 2005, 2: 8-10.1186/1479-5868-2-8.CrossRefPubMedPubMedCentral
30.
go back to reference Pate RR, Ward DS, Saunders RP, Felton G, Dishman RK, Dowda M: Promotion of physical activity among high-school girls: a randomized controlled trial. Am J Public Health. 2005, 95: 1582-1587. 10.2105/AJPH.2004.045807.CrossRefPubMedPubMedCentral Pate RR, Ward DS, Saunders RP, Felton G, Dishman RK, Dowda M: Promotion of physical activity among high-school girls: a randomized controlled trial. Am J Public Health. 2005, 95: 1582-1587. 10.2105/AJPH.2004.045807.CrossRefPubMedPubMedCentral
31.
go back to reference Ganz PA, Farmer MM, Belman MJ, Garcia CA, Streja L, Dietrich AJ, Winchell C, Bastani R, Kahn KL: Results of a randomized controlled trial to increase colorectal cancer screening in a managed care health plan. Cancer. 2005, 104: 2072-2083. 10.1002/cncr.21434.CrossRefPubMed Ganz PA, Farmer MM, Belman MJ, Garcia CA, Streja L, Dietrich AJ, Winchell C, Bastani R, Kahn KL: Results of a randomized controlled trial to increase colorectal cancer screening in a managed care health plan. Cancer. 2005, 104: 2072-2083. 10.1002/cncr.21434.CrossRefPubMed
Metadata
Title
Comparison of population-averaged and cluster-specific models for the analysis of cluster randomized trials with missing binary outcomes: a simulation study
Authors
Jinhui Ma
Parminder Raina
Joseph Beyene
Lehana Thabane
Publication date
01-12-2013
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2013
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-13-9

Other articles of this Issue 1/2013

BMC Medical Research Methodology 1/2013 Go to the issue