Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2018

Open Access 01-12-2018 | Research article

Meta-analysis of binary outcomes via generalized linear mixed models: a simulation study

Authors: Ilyas Bakbergenuly, Elena Kulinskaya

Published in: BMC Medical Research Methodology | Issue 1/2018

Login to get access

Abstract

Background

Systematic reviews and meta-analyses of binary outcomes are widespread in all areas of application. The odds ratio, in particular, is by far the most popular effect measure. However, the standard meta-analysis of odds ratios using a random-effects model has a number of potential problems. An attractive alternative approach for the meta-analysis of binary outcomes uses a class of generalized linear mixed models (GLMMs). GLMMs are believed to overcome the problems of the standard random-effects model because they use a correct binomial-normal likelihood. However, this belief is based on theoretical considerations, and no sufficient simulations have assessed the performance of GLMMs in meta-analysis. This gap may be due to the computational complexity of these models and the resulting considerable time requirements.

Methods

The present study is the first to provide extensive simulations on the performance of four GLMM methods (models with fixed and random study effects and two conditional methods) for meta-analysis of odds ratios in comparison to the standard random effects model.

Results

In our simulations, the hypergeometric-normal model provided less biased estimation of the heterogeneity variance than the standard random-effects meta-analysis using the restricted maximum likelihood (REML) estimation when the data were sparse, but the REML method performed similarly for the point estimation of the odds ratio, and better for the interval estimation.

Conclusions

It is difficult to recommend the use of GLMMs in the practice of meta-analysis. The problem of finding uniformly good methods of the meta-analysis for binary outcomes is still open.
Appendix
Available only for authorised users
Literature
1.
Higgins J, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc Ser A (Stat Soc). 2009; 172(1):137–59.CrossRef
2.
Mosteller F, Colditz GA. Understanding research synthesis (meta-analysis). Annu Rev Public Health. 1996; 17(1):1–23.CrossRefPubMed
3.
Stijnen T, Hamza TH, Özdemir P. Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Stat Med. 2010; 29(29):3046–67.CrossRefPubMed
4.
Kulinskaya E, Morgenthaler S, Staudte RG. Combining statistical evidence. Int Stat Rev. 2014; 82(2):214–42.CrossRef
5.
6.
Bakbergenuly I, Kulinskaya E, Morgenthaler S. Inference for binomial probability based on dependent Bernoulli random variables with applications to meta-analysis and group level studies. Biom J. 2016; 58(4):896–914.CrossRefPubMedPubMedCentral
7.
Viechtbauer W. Package metafor. The Comprehensive R Archive Network. Package ‘metafor’. http://​cran.​r-project.​org/​web/​packages/​metafor/​metafor.​pdf.​ 2017. Accessed 19 Feb 2017.
8.
Lee KJ, Thompson SG. Flexible parametric models for random-effects distributions. Stat Med. 2008; 27:418–34.CrossRefPubMed
9.
Kuss O. Statistical methods for meta-analyses including information from studies without any events—add nothing to nothing and succeed nevertheless. Stat Med. 2015; 34(7):1097–116.CrossRefPubMed
10.
11.
Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol. 2014; 47:881–9.CrossRef
12.
Sackett DL, Deeks JJ, Altman DG. Down with odds ratios!. Evid-Based Med. 1996; 1:164–6.
13.
14.
Deeks JJ. Issues in the selection of a summary statistic for meta-analysis of clinical trials with binary outcomes. Stat Med. 2002; 21:1575–600.CrossRefPubMed
15.
Newcombe RG. A deficiency of the odds ratio as a measure of effect size. Stat Med. 2006; 25:4235–40.CrossRefPubMed
16.
Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG. A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Stat Med. 2000; 19(24):3417–32.CrossRefPubMed
17.
Van Houwelingen HC, Zwinderman KH, Stijnen T. A bivariate approach to meta-analysis. Stat Med. 1993; 12(24):2273–84.CrossRefPubMed
18.
Liu Q, Pierce DA. Heterogeneity in Mantel-Haenszel-type models. Biometrika. 1993; 80(3):543–56.CrossRef
19.
Sidik K, Jonkman JN. Estimation using non-central hypergeometric distributions in combining 2 × 2 tables. J Stat Plan Infer. 2008; 138(12):3993–4005.CrossRef
20.
DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7(3):177–88.CrossRefPubMed
21.
Jackson D, Law M, Stijnen T, Viechtbauer W, White IR. A comparison of 7 random-effects models for meta-analyses that estimate the summary odds ratio. Stat Med. 2018; 37(7):1059–85.CrossRefPubMedPubMedCentral
22.
Breslow NE, Clayton DG. Approximate inference in generalized linear mixed models. J Am Stat Assoc. 1993; 88(421):9–25.
23.
24.
25.
Platt RW, Leroux BG, Breslow N. Generalized linear mixed models for meta-analysis. Stat Med. 1999; 18(6):643–54.CrossRefPubMed
26.
Gao S. Combining binomial data using the logistic normal model. J Stat Comput Simul. 2004; 74(4):293–306.CrossRef
27.
Hamza TH, Van Houwelingen HC, Stijnen T. The binomial distribution of meta-analysis was preferred to model within-study variability. J Clin Epidemiol. 2008; 61(1):41–51.CrossRefPubMed
28.
Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986; 73(1):13–22.CrossRef
29.
Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010; 36(3):1–48.CrossRef
30.
Liang KY. Odds ratio inference with dependent data. Biometrika. 1985; 72(3):678–82.CrossRef
31.
Fog A, Fog MA. The BiasedUrn package in R. 2015. http://​cran.​r-project.​org/​web/​packages/​BiasedUrn/​BiasedUrn.​pdf.​ Accessed 19 Feb 2017.
32.
Martin AD, Quinn KM, Park JH, Park MJH. The MCMCpack package in R. 2016. https://​cran.​r-project.​org/​web/​packages/​MCMCpack/​MCMCpack.​pdf.​ Accessed 19 Feb 2017.
33.
Gay DM. Usage summary for selected optimization routines. Comput Sci Tech Rep. 1990; 153:1–21.
34.
Powell MJ. The bobyqa algorithm for bound constrained optimization without derivatives. Cambridge: Cambridge NA Report NA2009/06, University of Cambridge; 2009. pp. 26–46.
35.
Viechtbauer W. Confidence intervals for the amount of heterogeneity in meta-analysis. Stat Med. 2007; 26(1):37–52.CrossRefPubMed
36.
Collins R, Yusuf S, Peto R. Overview of randomised trials of diuretics in pregnancy. Br Med J (Clin Res Ed). 1985; 290(6461):17–23.CrossRef
37.
Hardy RJ, Thompson SG. A likelihood approach to meta-analysis with random effects. Stat Med. 1996; 15(6):619–29.CrossRefPubMed
38.
Biggerstaff BJ, Tweedie RL. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Stat Med. 1997; 16(7):753–68.CrossRefPubMed
39.
Kulinskaya E, Olkin I. An overdispersion model in meta-analysis. Stat Model. 2014; 14(1):49–76.CrossRef
40.
Viechtbauer W. Bias and efficiency of meta-analytic variance estimators in the random-effects model. J Educ Behav Stat. 2005; 30(3):261–93.CrossRef
41.
Nemes S, Jonasson J, Genell A, Steineck G. Bias in odds ratios by logistic regression modelling and sample size. BMC Med Res Methodol. 2009; 9(1):1.CrossRef
42.
Kosmidis I, Guolo A, Varin C. Improving the accuracy of likelihood-based inference in meta-analysis and meta-regression. Biometrika. 2017; 104(2):489–96.
Metadata
Title
Meta-analysis of binary outcomes via generalized linear mixed models: a simulation study
Authors
Ilyas Bakbergenuly
Elena Kulinskaya
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2018
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-018-0531-9

Other articles of this Issue 1/2018

BMC Medical Research Methodology 1/2018 Go to the issue

Research article

Risks and rewards of using prepaid vs. postpaid incentive checks on a survey of physicians

Research article

Is recession bad for your mental health? The answer could be complex: evidence from the 2008 crisis in Spain

Research article

Estimating causal effects of time-dependent exposures on a binary endpoint in a high-dimensional setting

Research article

Application of the dyadic data analysis in behavioral medicine research: marital satisfaction and anxiety in infertile couples

Research article

Instrumental variable analysis in the context of dichotomous outcome and exposure with a numerical experiment in pharmacoepidemiology

Research article

Which ICD-9-CM codes should be used for bronchiolitis research?