Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2016

Open Access 01-12-2016 | Research Article

Performance of models for estimating absolute risk difference in multicenter trials with binary outcome

Authors: Claudia Pedroza, Van Thi Truong

Published in: BMC Medical Research Methodology | Issue 1/2016

Login to get access

Abstract

Background

Reporting of absolute risk difference (RD) is recommended for clinical and epidemiological prospective studies. In analyses of multicenter studies, adjustment for center is necessary when randomization is stratified by center or when there is large variation in patients outcomes across centers. While regression methods are used to estimate RD adjusted for baseline predictors and clustering, no formal evaluation of their performance has been previously conducted.

Methods

We performed a simulation study to evaluate 6 regression methods fitted under a generalized estimating equation framework: binomial identity, Poisson identity, Normal identity, log binomial, log Poisson, and logistic regression model. We compared the model estimates to unadjusted estimates. We varied the true response function (identity or log), number of subjects per center, true risk difference, control outcome rate, effect of baseline predictor, and intracenter correlation. We compared the models in terms of convergence, absolute bias and coverage of 95 % confidence intervals for RD.

Results

The 6 models performed very similar to each other for the majority of scenarios. However, the log binomial model did not converge for a large portion of the scenarios including a baseline predictor. In scenarios with outcome rate close to the parameter boundary, the binomial and Poisson identity models had the best performance, but differences from other models were negligible. The unadjusted method introduced little bias to the RD estimates, but its coverage was larger than the nominal value in some scenarios with an identity response. Under the log response, coverage from the unadjusted method was well below the nominal value (<80 %) for some scenarios.

Conclusions

We recommend the use of a binomial or Poisson GEE model with identity link to estimate RD for correlated binary outcome data. If these models fail to run, then either a logistic regression, log Poisson regression, or linear regression GEE model can be used.
Appendix
Available only for authorised users
Literature
1.
Greenland S. Interpretation and choice of effect measures in epidemiologic analyses. Am J Epidemiol. 1987; 125:761–8.PubMed
2.
Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ. 1995; 310:452–4.PubMedPubMedCentral
3.
Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol. 1994; 47:881–9.PubMed
4.
Schechtman E. Odds ratio, relative risk, absolute risk reduction, and the number needed to treat–which of these should we use?Value Health. 2002; 5:431–6.PubMed
5.
Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988; 318:1728–33.PubMed
6.
North D. Number needed to treat. Absolute risk reduction may be easier for patients to understand. BMJ. 1995; 310:1269.PubMedPubMedCentral
7.
Ayton P. Number needed to treat. Risk measures expressed as frequencies may have a more rational response. BMJ. 1995; 310:1269.PubMedPubMedCentral
8.
Hayward RA, Kent DM, Vijan S, Hofer TP. Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis. BMC Med Res Methodol. 2006; 6:18.PubMedPubMedCentral
9.
Fagerlin A, Zikmund-Fisher BJ, Ubel PA. Helping patients decide: ten steps to better risk communication. J Natl Cancer Inst. 2011; 103:1436–43.PubMedPubMedCentral
10.
Ghosh AK, Ghosh K. Translating evidence-based information into effective risk communication: current challenges and opportunities. J Lab Clin Med. 2005; 145:171–80.PubMed
11.
12.
Senn S. Testing for baseline balance in clinical trials. Stat Med. 1994; 13:1715–26.PubMed
13.
Kahan BC. Accounting for centre-effects in multicentre trials with a binary outcome–when, why, and how?BMC Med Res Methodol. 2014; 14:20.PubMedPubMedCentral
14.
Kahan BC, Morris TP. Improper analysis of trials randomised using stratified blocks or minimisation. Stat Med. 2012; 31:328–40.PubMed
15.
Parzen M, Lipsitz SR, Dear KB. Does clustering affect the usual test statistics of no treatment effect in a randomized clinical trial?Biom J. 1998; 40:385–02.
16.
Zou G. A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol. 2004; 159:702–6.PubMed
17.
Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. Am J Epidemiol. 2005; 162:199–200.PubMed
18.
Bender R, Kuss O, Hildebrandt M, Gehrmann U. Estimating adjusted NNT measures in logistic regression analysis. Stat Med. 2007; 26:5586–95.PubMed
19.
Gehrmann U, Kuss O, Wellmann J, Bender R. Logistic regression was preferred to estimate risk differences and numbers needed to be exposed adjusted for covariates. J Clin Epidemiol. 2010; 63:1223–31.PubMed
20.
Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986; 73:13–22.
21.
Mancl LA, DeRouen TA. A covariance estimator for GEE with improved small-sample properties. Biometrics. 2001; 57:126–34.PubMed
22.
Shankaran S, Laptook AR, Pappas A, McDonald SA, Das A, Tyson JE, et al. Effect of depth and duration of cooling on deaths in the NICU among neonates with hypoxic ischemic encephalopathy: a randomized clinical trial. JAMA. 2014; 312:2629–39.PubMedPubMedCentral
23.
Rouse DJ, Hirtz DG, Thom E, Varner MW, Spong CY, Mercer BM, et al. A randomized, controlled trial of magnesium sulfate for the prevention of cerebral palsy. N Engl J Med. 2008; 359:895–905.PubMedPubMedCentral
24.
Morris BH, Oh W, Tyson JE, Stevenson DK, Phelps DL, et al. Aggressive vs. conservative phototherapy for infants with extremely low birth weight. N Engl J Med. 2008; 359:1885–96.PubMedPubMedCentral
25.
Eldridge SM, Ashby D, Feder GS, Rudnicka AR. Ukoumunne OC. Lessons for cluster randomized trials in the twenty-first century: a systematic review of trials in primary care. Clin Trials. 2004; 1:80–90.PubMed
26.
Austin PC. A comparison of the statistical power of different methods for the analysis of cluster randomization trials with binary outcomes. Stat Med. 2007; 26:3550–65.PubMed
27.
Thompson SG, Warn DE, Turner RM. Bayesian methods for analysis of binary outcome data in cluster randomized trials on the absolute risk scale. Stat Med. 2004; 23:389–410.PubMed
28.
Turner RM, Omar RZ, Thompson SG. Bayesian methods of analysis for cluster randomized trials with binary outcome data. Stat Med. 2001; 20:453–72.PubMed
29.
30.
Hjsgaard S, Halekoh U, Yan J. The R Package geepack for Generalized Estimating Equations. J Stat Softw. 2006; 15/2:1–11.
31.
Yan J, Fine JP. Estimating Equations for Association Structures. Stat Med. 2004; 23:859–80.PubMed
32.
Yan J. geepack: Yet Another Package for Generalized Estimating Equations. R-News. 2002; 2/3:12–4.
33.
34.
Moher M, Yudkin P, Wright L, Turner R, Fuller A, Schofield T, et al. Cluster randomised controlled trial to compare three methods of promoting secondary prevention of coronary heart disease in primary care. BMJ. 2001; 322:1338.PubMedPubMedCentral
35.
Beitler PJ, Landis JR. A mixed-effects model for categorical data. Biometrics. 1985; 41:991–1000.PubMed
36.
Rothwell PM, Mehta Z, Howard SC, Gutnikov SA, Warlow CP. Treating individuals 3: from subgroups to individuals: general principles and the example of carotid endarterectomy. Lancet. 2005; 365:256–65.PubMed
37.
Ukoumunne OC, Forbes AB, Carlin JB, Gulliford MC. Comparison of the risk difference, risk ratio and odds ratio scales for quantifying the unadjusted intervention effect in cluster randomized trials. Stat Med. 2008; 27:5143–55.PubMed
38.
Cheung YB. A modified least-squares regression approach to the estimation of risk difference. Am J Epidemiol. 2007; 166:1337–44.PubMed
39.
Yelland LN, Salter AB, Ryan P. Performance of the modified Poisson regression approach for estimating relative risks from clustered prospective data. Am J Epidemiol. 2011; 174:984–92.PubMed
40.
Zou GY, Donner A. Extension of the modified Poisson regression model to prospective studies with correlated binary data. Stat Methods Med Res. 2013; 22:661–70.PubMed
41.
Cheng G, Yu Z, Huang JZ. The cluster bootstrap consistency in generalized estimating equations. J Multivar Anal. 2013; 115:33–47.
42.
Yelland LN, Salter AB, Ryan P, Laurence CO. Adjusted intraclass correlation coefficients for binary data: methods and estimates from a cluster-randomized trial in primary care. Clin Trials. 2011; 8:48–58.PubMed
43.
Ritz J, Spiegelman D. Equivalence of conditional and marginal regression models for clustered and longitudinal data. Stat Methods Med Res. 2004; 13:309–23.
44.
Diggle P, Heagerty P, Liang KY, Zeger S. Analysis of Longitudinal Data: Oxford: Oxford University Press; 2002.
45.
Li P, Redden DT. Small sample performance of bias-corrected sandwich estimators for cluster-randomized trials with binary outcomes. Stat Med. 2015; 34:281–96.PubMed
46.
Kahan BC, Harhay MO. Many multicenter trials had few events per center, requiring analysis via random-effects models or GEEs. J Clin Epidemiol. 2015; 68:1504–11.PubMedPubMedCentral
47.
Tyson J, Pedroza C, Wootton SH. Analyses of Large Data Bases and Pragmatic Clinical Trials: Advancing Comparative Effectiveness Research in a Learning Health Care System. Am J Perinatol. 2015; 32:1095–7.PubMed
Metadata
Title
Performance of models for estimating absolute risk difference in multicenter trials with binary outcome
Authors
Claudia Pedroza
Van Thi Truong
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0217-0

Other articles of this Issue 1/2016

BMC Medical Research Methodology 1/2016 Go to the issue

Research article

Factors relating to participation in follow-up to the 45 and up study in Aboriginal and non-Aboriginal individuals

Research article

Time to publication among completed diagnostic accuracy studies: associated with reported accuracy estimates

Research article

Patient enrollment and logistical problems top the list of difficulties in clinical research: a cross-sectional survey

Research article

Intention-to-treat and transparency of related practices in randomized, controlled trials of anti-infectives

Erratum

Erratum to: Assessment of predictive performance in incomplete data by combining internal validation and multiple imputation

Research article

Potential use of telephone surveys for non-communicable disease surveillance in developing countries: evidence from a national household survey in Lebanon