Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2016

Open Access 01-12-2016 | Research Article

Two new methods to fit models for network meta-analysis with random inconsistency effects

Authors: Martin Law, Dan Jackson, Rebecca Turner, Kirsty Rhodes, Wolfgang Viechtbauer

Published in: BMC Medical Research Methodology | Issue 1/2016

Login to get access

Abstract

Background

Meta-analysis is a valuable tool for combining evidence from multiple studies. Network meta-analysis is becoming more widely used as a means to compare multiple treatments in the same analysis. However, a network meta-analysis may exhibit inconsistency, whereby the treatment effect estimates do not agree across all trial designs, even after taking between-study heterogeneity into account. We propose two new estimation methods for network meta-analysis models with random inconsistency effects.

Methods

The model we consider is an extension of the conventional random-effects model for meta-analysis to the network meta-analysis setting and allows for potential inconsistency using random inconsistency effects. Our first new estimation method uses a Bayesian framework with empirically-based prior distributions for both the heterogeneity and the inconsistency variances. We fit the model using importance sampling and thereby avoid some of the difficulties that might be associated with using Markov Chain Monte Carlo (MCMC). However, we confirm the accuracy of our importance sampling method by comparing the results to those obtained using MCMC as the gold standard. The second new estimation method we describe uses a likelihood-based approach, implemented in the metafor package, which can be used to obtain (restricted) maximum-likelihood estimates of the model parameters and profile likelihood confidence intervals of the variance components.

Results

We illustrate the application of the methods using two contrasting examples. The first uses all-cause mortality as an outcome, and shows little evidence of between-study heterogeneity or inconsistency. The second uses “ear discharge" as an outcome, and exhibits substantial between-study heterogeneity and inconsistency. Both new estimation methods give results similar to those obtained using MCMC.

Conclusions

The extent of heterogeneity and inconsistency should be assessed and reported in any network meta-analysis. Our two new methods can be used to fit models for network meta-analysis with random inconsistency effects. They are easily implemented using the accompanying R code in the Additional file 1. Using these estimation methods, the extent of inconsistency can be assessed and reported.
Appendix
Available only for authorised users
Literature
1.
go back to reference DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7:177–88. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986; 7:177–88.
2.
go back to reference Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012; 3:80–97. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012; 3:80–97.
3.
go back to reference Salanti G, Higgins JPT, Ades AE, Ioannidis JPA. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008; 17:279–301. Salanti G, Higgins JPT, Ades AE, Ioannidis JPA. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008; 17:279–301.
4.
go back to reference Dias S, Ades AE. Absolute or relative effects? Arm-based synthesis of trial data. Res Synth Methods. 2016; 7(1):23–8.CrossRefPubMed Dias S, Ades AE. Absolute or relative effects? Arm-based synthesis of trial data. Res Synth Methods. 2016; 7(1):23–8.CrossRefPubMed
5.
go back to reference Hong H, Chu H, Zhang J, Carlin BP. Rejoinder to the discussion of “a Bayesian missing data framework for generalized multiple outcome mixed treatment comparisons,” by S. Dias and A.E. Ades. Res Synth Methods. 2016; 7(1):29–33.CrossRefPubMed Hong H, Chu H, Zhang J, Carlin BP. Rejoinder to the discussion of “a Bayesian missing data framework for generalized multiple outcome mixed treatment comparisons,” by S. Dias and A.E. Ades. Res Synth Methods. 2016; 7(1):29–33.CrossRefPubMed
6.
go back to reference Piepho HP, Williams ER, Madden LV. The use of two-way linear mixed models in multitreatment meta-analysis. Biometrics. 2012; 68:1269–77. Piepho HP, Williams ER, Madden LV. The use of two-way linear mixed models in multitreatment meta-analysis. Biometrics. 2012; 68:1269–77.
7.
go back to reference Jackson D, Law M, Barrett JK, Turner R, Higgins JPT, Salanti G, White IR. Extending DerSimonian and Laird’s methodology to perform network meta-analysis with random inconsistency effects. Stat Med. 2016; 35(6):819–39.CrossRefPubMed Jackson D, Law M, Barrett JK, Turner R, Higgins JPT, Salanti G, White IR. Extending DerSimonian and Laird’s methodology to perform network meta-analysis with random inconsistency effects. Stat Med. 2016; 35(6):819–39.CrossRefPubMed
8.
go back to reference Jackson D, Barrett JK, Rice S, White IR, Higgins JPT. A design-by-treatment interaction model for network meta-analysis with random inconsistency effects. Stat Med. 2014; 33:3639–54. Jackson D, Barrett JK, Rice S, White IR, Higgins JPT. A design-by-treatment interaction model for network meta-analysis with random inconsistency effects. Stat Med. 2014; 33:3639–54.
9.
go back to reference Lunn DJ, Thomas A, Best N, Spiegelhalter D. WinBUGS– a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. 2000; 10:325–37. Lunn DJ, Thomas A, Best N, Spiegelhalter D. WinBUGS– a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. 2000; 10:325–37.
10.
go back to reference Krahn U, Binder H, König J. A graphical tool for locating inconsistency in network meta-analyses. BMC Med Res Methodol. 2013; 13:35. Krahn U, Binder H, König J. A graphical tool for locating inconsistency in network meta-analyses. BMC Med Res Methodol. 2013; 13:35.
11.
go back to reference Lambert PC, Sutton AJ, Burton PR, Abrams KR, Jones DR. How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Stat Med. 2005; 24:2401–28. Lambert PC, Sutton AJ, Burton PR, Abrams KR, Jones DR. How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS. Stat Med. 2005; 24:2401–28.
12.
go back to reference White IR, Barrett JK, Jackson D, Higgins JPT. Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Res Synth Methods. 2012; 3:111–25. White IR, Barrett JK, Jackson D, Higgins JPT. Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Res Synth Methods. 2012; 3:111–25.
13.
go back to reference Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Synth Methods. 2012; 3:98–110. Higgins JPT, Jackson D, Barrett JK, Lu G, Ades AE, White IR. Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Synth Methods. 2012; 3:98–110.
14.
go back to reference Lu G, Ades AE. Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc. 2006; 101:447–59. Lu G, Ades AE. Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc. 2006; 101:447–59.
16.
go back to reference Piepho HP. Network meta-analysis made easy: detection of inconsistency using factorial analysis-of-variance models. BMC Med Res Methodol. 2014; 14:61. Piepho HP. Network meta-analysis made easy: detection of inconsistency using factorial analysis-of-variance models. BMC Med Res Methodol. 2014; 14:61.
17.
go back to reference Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med. 2002; 21:2313–24. Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med. 2002; 21:2313–24.
18.
go back to reference Jackson D, Riley R, White IR. Multivariate meta-analysis: Potential and promise. Stat Med. 2011; 30:2481–98. Jackson D, Riley R, White IR. Multivariate meta-analysis: Potential and promise. Stat Med. 2011; 30:2481–98.
19.
go back to reference Turner RM, Jackson D, Wei Y, Thompson SG, Higgins JPT. Predictive distributions for between-study heterogeneity and methods for their application in Bayesian meta-analysis. Stat Med. 2014; 34:984–98. Turner RM, Jackson D, Wei Y, Thompson SG, Higgins JPT. Predictive distributions for between-study heterogeneity and methods for their application in Bayesian meta-analysis. Stat Med. 2014; 34:984–98.
20.
go back to reference Whitehead A. Meta-analysis of Controlled Clinical Trials, 1st edition. New York: John Wiley & Sons; 2002, p. 282.CrossRef Whitehead A. Meta-analysis of Controlled Clinical Trials, 1st edition. New York: John Wiley & Sons; 2002, p. 282.CrossRef
21.
go back to reference O’Hagan, Kendall FJ. Advanced Theory of Statistics, vol 2B. London: Bayesian Inference. Arnold; 2004. O’Hagan, Kendall FJ. Advanced Theory of Statistics, vol 2B. London: Bayesian Inference. Arnold; 2004.
22.
go back to reference Baker R, Jackson D. A new approach to outliers in meta-analysis. Health Care Manag Sci. 2008; 11:121–31. Baker R, Jackson D. A new approach to outliers in meta-analysis. Health Care Manag Sci. 2008; 11:121–31.
24.
25.
go back to reference Hardy RJ, Thompson SG. A likelihood approach to meta-analysis with random effects. Stat Med. 1996; 15:619–29. Hardy RJ, Thompson SG. A likelihood approach to meta-analysis with random effects. Stat Med. 1996; 15:619–29.
26.
go back to reference White IR. Network meta-analysis. Stata J. 2015; 15(4):951–85. White IR. Network meta-analysis. Stata J. 2015; 15(4):951–85.
27.
go back to reference Chaimani A, Salanti G. Visualizing assumptions and results in network meta-analysis: The network graphs package. Stata J. 2015; 15(4):905–50. Chaimani A, Salanti G. Visualizing assumptions and results in network meta-analysis: The network graphs package. Stata J. 2015; 15(4):905–50.
29.
go back to reference Xiong T, Turner R, Wei Y, Neal DE, Lyratzopoulos G, Higgins JPT. Comparative efficacy and safety of treatments for localized prostate cancer: an application of network meta-analysis. BMJ Open. 2014; 4:e004285. Xiong T, Turner R, Wei Y, Neal DE, Lyratzopoulos G, Higgins JPT. Comparative efficacy and safety of treatments for localized prostate cancer: an application of network meta-analysis. BMJ Open. 2014; 4:e004285.
30.
go back to reference Macfadyen CA, Acuin JM, Gamble C. Topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. Cochrane Database Syst Rev. 2005; 4:CD004618. Macfadyen CA, Acuin JM, Gamble C. Topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. Cochrane Database Syst Rev. 2005; 4:CD004618.
31.
go back to reference Gumedze FN, Jackson D. A random effects variance shift model for detecting and accommodating outliers in meta-analysis. BMC Med Res Methodol. 2011; 11:19. Gumedze FN, Jackson D. A random effects variance shift model for detecting and accommodating outliers in meta-analysis. BMC Med Res Methodol. 2011; 11:19.
32.
go back to reference Jackson D, Turner R, Rhodes K, Viechtbauer W. Methods for calculating confidence and credible intervals for the residual between-study variance in random effects meta-regression models. BMC Med Res Methodol. 2014; 14:103. Jackson D, Turner R, Rhodes K, Viechtbauer W. Methods for calculating confidence and credible intervals for the residual between-study variance in random effects meta-regression models. BMC Med Res Methodol. 2014; 14:103.
Metadata
Title
Two new methods to fit models for network meta-analysis with random inconsistency effects
Authors
Martin Law
Dan Jackson
Rebecca Turner
Kirsty Rhodes
Wolfgang Viechtbauer
Publication date
01-12-2016
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0184-5

Other articles of this Issue 1/2016

BMC Medical Research Methodology 1/2016 Go to the issue