Skip to main content
Top
Published in: Systematic Reviews 1/2021

Open Access 01-12-2021 | Macrolide | Methodology

A likelihood ratio test for the homogeneity of between-study variance in network meta-analysis

Authors: Dapeng Hu, Chong Wang, Annette M. O’Connor

Published in: Systematic Reviews | Issue 1/2021

Login to get access

Abstract

Background

Network meta-analysis (NMA) is a statistical method used to combine results from several clinical trials and simultaneously compare multiple treatments using direct and indirect evidence. Statistical heterogeneity is a characteristic describing the variability in the intervention effects being evaluated in the different studies in network meta-analysis. One approach to dealing with statistical heterogeneity is to perform a random effects network meta-analysis that incorporates a between-study variance into the statistical model. A common assumption in the random effects model for network meta-analysis is the homogeneity of between-study variance across all interventions. However, there are applications of NMA where the single between-study assumption is potentially incorrect and instead the model should incorporate more than one between-study variances.

Methods

In this paper, we develop an approach to testing the homogeneity of between-study variance assumption based on a likelihood ratio test. A simulation study was conducted to assess the type I error and power of the proposed test. This method is then applied to a network meta-analysis of antibiotic treatments for Bovine respiratory disease (BRD).

Results

The type I error rate was well controlled in the Monte Carlo simulation. We found statistical evidence (p value = 0.052) against the homogeneous between-study variance assumption in the network meta-analysis BRD. The point estimate and confidence interval of relative effect sizes are strongly influenced by this assumption.

Conclusions

Since homogeneous between-study variance assumption is a strong assumption, it is crucial to test the validity of this assumption before conducting a network meta-analysis. Here we propose and validate a method for testing this single between-study variance assumption which is widely used for many NMA.
Literature
1.
go back to reference Song F, Altman DG, Glenny A-M, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ. 2003; 326(7387):472.CrossRef Song F, Altman DG, Glenny A-M, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ. 2003; 326(7387):472.CrossRef
2.
go back to reference Glenny A, Altman D, Song F, Sakarovitch C, Deeks J, D’Amico R, Bradburn M, Eastwood A, et al.Indirect comparisons of competing interventions. Health Technol Assess. 2005; 9(26):1–134.CrossRef Glenny A, Altman D, Song F, Sakarovitch C, Deeks J, D’Amico R, Bradburn M, Eastwood A, et al.Indirect comparisons of competing interventions. Health Technol Assess. 2005; 9(26):1–134.CrossRef
3.
go back to reference Dias S, Ades AE, Welton NJ, Jansen JP, Sutton AJ. Network Meta-analysis for Decision-making. Hoboken: John Wiley & Sons; 2018.CrossRef Dias S, Ades AE, Welton NJ, Jansen JP, Sutton AJ. Network Meta-analysis for Decision-making. Hoboken: John Wiley & Sons; 2018.CrossRef
4.
go back to reference Dias S, Welton NJ, Sutton AJ, Ades A. Nice dsu technical support document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. 2011. https://www.nicedsu.org.uk. Dias S, Welton NJ, Sutton AJ, Ades A. Nice dsu technical support document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials. 2011. https://​www.​nicedsu.​org.​uk.
5.
go back to reference Lu G, Ades A. Modeling between-trial variance structure in mixed treatment comparisons. Biostatistics. 2009; 10(4):792–805.CrossRef Lu G, Ades A. Modeling between-trial variance structure in mixed treatment comparisons. Biostatistics. 2009; 10(4):792–805.CrossRef
6.
go back to reference White IR, Barrett JK, Jackson D, Higgins JP. Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Res Synth Methods. 2012; 3(2):111–25.CrossRef White IR, Barrett JK, Jackson D, Higgins JP. Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Res Synth Methods. 2012; 3(2):111–25.CrossRef
7.
go back to reference Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JP. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the cochrane database of systematic reviews. Int J Epidemiol. 2012; 41(3):818–27.CrossRef Turner RM, Davey J, Clarke MJ, Thompson SG, Higgins JP. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the cochrane database of systematic reviews. Int J Epidemiol. 2012; 41(3):818–27.CrossRef
8.
go back to reference Turner RM, Domínguez-Islas CP, Jackson D, Rhodes KM, White IR. Incorporating external evidence on between-trial heterogeneity in network meta-analysis. Stat Med. 2019; 38(8):1321–35.CrossRef Turner RM, Domínguez-Islas CP, Jackson D, Rhodes KM, White IR. Incorporating external evidence on between-trial heterogeneity in network meta-analysis. Stat Med. 2019; 38(8):1321–35.CrossRef
9.
go back to reference DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7(3):177–88.CrossRef DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7(3):177–88.CrossRef
10.
go back to reference Kontopantelis E, Springate DA, Reeves D. A re-analysis of the cochrane library data: the dangers of unobserved heterogeneity in meta-analyses. PloS ONE. 2013; 8(7):69930.CrossRef Kontopantelis E, Springate DA, Reeves D. A re-analysis of the cochrane library data: the dangers of unobserved heterogeneity in meta-analyses. PloS ONE. 2013; 8(7):69930.CrossRef
11.
go back to reference Hunter JE, Schmidt FL. Methods of Meta-analysis: Correcting Error and Bias in Research Findings. London: Sage; 2004.CrossRef Hunter JE, Schmidt FL. Methods of Meta-analysis: Correcting Error and Bias in Research Findings. London: Sage; 2004.CrossRef
12.
go back to reference O’Connor AM, Hu D, Totton SC, Scott N, Winder CB, Wang B, Wang C, Glanville J, Wood H, White B, et al. A systematic review and network meta-analysis of injectable antibiotic options for the control of bovine respiratory disease in the first 45 days post arrival at the feedlot. Anim Health Res Rev. 2019; 20(2):163–81.CrossRef O’Connor AM, Hu D, Totton SC, Scott N, Winder CB, Wang B, Wang C, Glanville J, Wood H, White B, et al. A systematic review and network meta-analysis of injectable antibiotic options for the control of bovine respiratory disease in the first 45 days post arrival at the feedlot. Anim Health Res Rev. 2019; 20(2):163–81.CrossRef
13.
go back to reference Abell KM, Theurer ME, Larson RL, White BJ, Apley M. A mixed treatment comparison meta-analysis of metaphylaxis treatments for bovine respiratory disease in beef cattle, 2. J Anim Sci. 2017; 95(2):626–35.PubMed Abell KM, Theurer ME, Larson RL, White BJ, Apley M. A mixed treatment comparison meta-analysis of metaphylaxis treatments for bovine respiratory disease in beef cattle, 2. J Anim Sci. 2017; 95(2):626–35.PubMed
14.
go back to reference Salanti G, Higgins JP, Ades A, Ioannidis JP. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008; 17(3):279–301.CrossRef Salanti G, Higgins JP, Ades A, Ioannidis JP. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008; 17(3):279–301.CrossRef
15.
go back to reference O’Connor A, Yuan C, Cullen J, Coetzee J, Da Silva N, Wang C. A mixed treatment meta-analysis of antibiotic treatment options for bovine respiratory disease–an update. Prev Vet Med. 2016; 132:130–39.CrossRef O’Connor A, Yuan C, Cullen J, Coetzee J, Da Silva N, Wang C. A mixed treatment meta-analysis of antibiotic treatment options for bovine respiratory disease–an update. Prev Vet Med. 2016; 132:130–39.CrossRef
16.
go back to reference Jackson D, Veroniki AA, Law M, Tricco AC, Baker R. Paule-mandel estimators for network meta-analysis with random inconsistency effects. Res Synth Methods. 2017; 8(4):416–34.CrossRef Jackson D, Veroniki AA, Law M, Tricco AC, Baker R. Paule-mandel estimators for network meta-analysis with random inconsistency effects. Res Synth Methods. 2017; 8(4):416–34.CrossRef
17.
go back to reference van Aert RC, Jackson D. Multistep estimators of the between-study variance: The relationship with the paule-mandel estimator. Stat Med. 2018; 37(17):2616–29.CrossRef van Aert RC, Jackson D. Multistep estimators of the between-study variance: The relationship with the paule-mandel estimator. Stat Med. 2018; 37(17):2616–29.CrossRef
18.
go back to reference Langan D, Higgins JP, Simmonds M. Comparative performance of heterogeneity variance estimators in meta-analysis: a review of simulation studies. Res Synth Methods. 2017; 8(2):181–98.CrossRef Langan D, Higgins JP, Simmonds M. Comparative performance of heterogeneity variance estimators in meta-analysis: a review of simulation studies. Res Synth Methods. 2017; 8(2):181–98.CrossRef
19.
go back to reference Langan D, Higgins JP, Jackson D, Bowden J, Veroniki AA, Kontopantelis E, Viechtbauer W, Simmonds M. A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Res Synth Methods. 2019; 10(1):83–98.CrossRef Langan D, Higgins JP, Jackson D, Bowden J, Veroniki AA, Kontopantelis E, Viechtbauer W, Simmonds M. A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Res Synth Methods. 2019; 10(1):83–98.CrossRef
Metadata
Title
A likelihood ratio test for the homogeneity of between-study variance in network meta-analysis
Authors
Dapeng Hu
Chong Wang
Annette M. O’Connor
Publication date
01-12-2021
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2021
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/s13643-021-01859-3

Other articles of this Issue 1/2021

Systematic Reviews 1/2021 Go to the issue