Top

Systematic Reviews

Published in:

Open Access 01-12-2012 | Methodology

Sample size and power considerations in network meta-analysis

Authors: Kristian Thorlund, Edward J Mills

Published in: Systematic Reviews | Issue 1/2012

Abstract

Background

Network meta-analysis is becoming increasingly popular for establishing comparative effectiveness among multiple interventions for the same disease. Network meta-analysis inherits all methodological challenges of standard pairwise meta-analysis, but with increased complexity due to the multitude of intervention comparisons. One issue that is now widely recognized in pairwise meta-analysis is the issue of sample size and statistical power. This issue, however, has so far only received little attention in network meta-analysis. To date, no approaches have been proposed for evaluating the adequacy of the sample size, and thus power, in a treatment network.

Findings

In this article, we develop easy-to-use flexible methods for estimating the ‘effective sample size’ in indirect comparison meta-analysis and network meta-analysis. The effective sample size for a particular treatment comparison can be interpreted as the number of patients in a pairwise meta-analysis that would provide the same degree and strength of evidence as that which is provided in the indirect comparison or network meta-analysis. We further develop methods for retrospectively estimating the statistical power for each comparison in a network meta-analysis. We illustrate the performance of the proposed methods for estimating effective sample size and statistical power using data from a network meta-analysis on interventions for smoking cessation including over 100 trials.

Conclusion

The proposed methods are easy to use and will be of high value to regulatory agencies and decision makers who must assess the strength of the evidence supporting comparative effectiveness estimates.

Available only for authorised users

Sutton AJ, Higgins JP: Recent developments in meta-analysis. Stat Med. 2008, 27: 625-650. 10.1002/sim.2934.CrossRefPubMed

Ioannidis JP: Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. Cmaj. 2009, 181: 488-493. 10.1503/cmaj.081086.CrossRefPubMedPubMedCentral

Jansen JP, Crawford B, Bergman G, Stam W: Bayesian meta-analysis of multiple treatment comparisons: an introduction to mixed treatment comparisons. Value Health. 2008, 11: 956-964. 10.1111/j.1524-4733.2008.00347.x.CrossRefPubMed

Lu G, Ades AE: Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med. 2004, 23: 3105-3124. 10.1002/sim.1875.CrossRefPubMed

Mills EJ, Bansback N, Ghement I, Thorlund K, Kelly S, Puhan MA, Wright J: Multiple treatment comparison meta-analyses: a step forward into complexity. Clin Epidemiol. 2011, 3: 193-202.CrossRefPubMedPubMedCentral

Salanti G, Higgins JP, Ades AE, Ioannidis JP: Evaluation of networks of randomized trials. Stat Methods Med Res. 2008, 17: 279-301.CrossRefPubMed

Sutton A, Ades AE, Cooper N, Abrams K: Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics. 2008, 26: 753-767. 10.2165/00019053-200826090-00006.CrossRefPubMed

Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks JJ, D'Amico R, Bradburn M, Eastwood AJ: Indirect comparisons of competing interventions. Health Technol Assess. 2005, 9: 1-134. iii-ivCrossRefPubMed

Song F, Altman DG, Glenny AM, Deeks JJ: Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ. 2003, 326: 472-10.1136/bmj.326.7387.472.CrossRefPubMedPubMedCentral

10.

Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG: Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ. 2009, 338: b1147-10.1136/bmj.b1147.CrossRefPubMedPubMedCentral

11.

Song F, Xiong T, Parekh-Bhurke S, Loke YK, Sutton AJ, Eastwood AJ, Holland R, Chen YF, Glenny AM, Deeks JJ, Altman DG: Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study. BMJ. 2011, 343: d4909-10.1136/bmj.d4909.CrossRefPubMedPubMedCentral

12.

Higgins JPT, Green S (editors): Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2 [updated September 2009]. The Cochrane Collaboration. 2009

13.

Mills EJ, Ioannidis JPA, Thorlund K, Schünemann HJ, Puhan MA, Guyatt GH: How to use an article reporting a multiple treatment comparison meta-analysis. JAMA. 2012, 308: 1246-53. 10.1001/2012.jama.11228.CrossRefPubMed

14.

Brok J, Thorlund K, Gluud C, Wetterslev J: Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses. J Clin Epidemiol. 2008, 61: 763-769. 10.1016/j.jclinepi.2007.10.007.CrossRefPubMed

15.

Brok J, Thorlund K, Wetterslev J, Gluud C: Apparently conclusive meta-analyses may be inconclusive–Trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses. Int J Epidemiol. 2009, 38: 287-298.CrossRefPubMed

16.

Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, Devereaux PJ, Montori VM, Freyschuss B, Vist G: GRADE guidelines 6. Rating the quality of evidence--imprecision. J Clin Epidemiol. 2011, 64: 1283-1293. 10.1016/j.jclinepi.2011.01.012.CrossRefPubMed

17.

Higgins JP, Whitehead A, Simmonds M: Sequential methods for random-effects meta-analysis. Stat Med. 2011, 30: 903-921. 10.1002/sim.4088.CrossRefPubMed

18.

Ioannidis J, Lau J: Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses. Proc Natl Acad Sci USA. 2001, 98: 831-836. 10.1073/pnas.98.3.831.CrossRefPubMedPubMedCentral

19.

Pogue J, Yusuf S: Overcoming the limitations of current meta-analysis of randomised controlled trials. Lancet. 1998, 351: 47-52. 10.1016/S0140-6736(97)08461-4.CrossRefPubMed

20.

Pogue JM, Yusuf S: Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Control Clin Trials. 1997, 18: 580-593. 10.1016/S0197-2456(97)00051-2. discussion 661–586CrossRefPubMed

21.

Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, Thabane L, Gluud LL, Als-Nielsen B, Gluud C: Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?. Int J Epidemiol. 2009, 38: 276-286.CrossRefPubMed

22.

Thorlund K, Imberger G, Walsh M, Chu R, Gluud C, Wetterslev J, Guyatt G, Devereaux PJ, Thabane L: The number of patients and events required to limit the risk of overestimation of intervention effects in meta-analysis–a simulation study. PLoS One. 2011, 6: e25491-10.1371/journal.pone.0025491.CrossRefPubMedPubMedCentral

23.

Wetterslev J, Thorlund K, Brok J, Gluud C: Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. J Clin Epidemiol. 2008, 61: 64-75. 10.1016/j.jclinepi.2007.03.013.CrossRefPubMed

24.

Wetterslev J, Thorlund K, Brok J, Gluud C: Estimating required information size by quantifying diversity in random-effects model meta-analyses. BMC Med Res Methodol. 2009, 9: 86-10.1186/1471-2288-9-86.CrossRefPubMedPubMedCentral

25.

van der Tweel I, Bollen C: Sequential meta-analysis: an efficient decision-making tool. Clin Trials. 2010, 7: 136-146. 10.1177/1740774509360994.CrossRefPubMed

26.

Thorlund K, Anema A, Mills E: Interpreting meta-analysis according to the adequacy of sample size. An example using isoniazid chemoprophylaxis for tuberculosis in purified protein derivative negative HIV-infected individuals. Clin Epidemiol. 2010, 2: 57-66.PubMedPubMedCentral

27.

Pereira TV, Ioannidis JP: Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. J Clin Epidemiol. 2011, 64: 1060-1069. 10.1016/j.jclinepi.2010.12.012.CrossRefPubMed

28.

Thorlund K, Engstrom J, Wetterslev J, Brok J, Imberger G, Gluud C: User manual for trial sequential analysis (TSA). In Book User manual for trial sequential analysis (TSA). 2011, Copenhagen Trial Unit, City

29.

Mills EJ, Ghement I, O'Regan C, Thorlund K: Estimating the power of indirect comparisons: a simulation study. PLoS One. 2011, 6: e16237-10.1371/journal.pone.0016237.CrossRefPubMedPubMedCentral

30.

Lumley T: Network meta-analysis for indirect treatment comparisons. Stat Med. 2002, 21: 2313-2324. 10.1002/sim.1201.CrossRefPubMed

31.

Mills EJ, Wu P, Lockhart I, Thorlund K, Puhan MA, Ebbert JO: Comparison of high-dose and combination nicotine replacement therapy, varenicline, and buproprion for smoking cessation: A systematic review and multiple treatment meta-analysis. Ann Med. 2012, 44: 588-97. 10.3109/07853890.2012.705016.CrossRefPubMed

32.

Bucher HC, Guyatt GH, Griffith LE, Walter SD: The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997, 50: 683-691. 10.1016/S0895-4356(97)00049-8.CrossRefPubMed

33.

Higgins JP, Whitehead A: Borrowing strength from external trials in a meta-analysis. Stat Med. 1996, 15: 2733-2749. 10.1002/(SICI)1097-0258(19961230)15:24<2733::AID-SIM562>3.0.CO;2-0.CrossRefPubMed

34.

Higgins JP, Thompson SG: Quantifying heterogeneity in a meta-analysis. Stat Med. 2002, 21: 1539-1558. 10.1002/sim.1186.CrossRefPubMed

35.

Borenstein M, Hedges L, Higgins JP: Introduction to meta-analysis. 2009, John Wiley and Sons, Chichester, 25CrossRef

36.

Thorlund K, Imberger G, Johnston B, Walsh M, Awad T, Thabane L, Gluud C, Devereaux PJ, Wetterslev J: Evolution of heterogeneity (I2) estimates and their 95% confidence intervals in large meta-analyses. PLoS One. 2012, 7: e39471-10.1371/journal.pone.0039471.CrossRefPubMedPubMedCentral

37.

Whitehead A, Whitehead J: A general parametric approach to the meta-analysis of randomized clinical trials. Stat Med. 1991, 10: 1665-1677. 10.1002/sim.4780101105.CrossRefPubMed

38.

Thorlund K, Imberger G, Wetterslev J, Brok J, Gluud C: Comments on 'Sequential meta-analysis: an efficient decision-making tool' by I van der Tweel and C Bollen. Clin Trials. 2010, 7: 752-753. 10.1177/1740774510386255. author reply 754CrossRefPubMed

Title: Sample size and power considerations in network meta-analysis
Authors: Kristian Thorlund
Edward J Mills
Publication date: 01-12-2012
Publisher: BioMed Central
Published in: Systematic Reviews / Issue 1/2012
Electronic ISSN: 2046-4053
DOI: https://doi.org/10.1186/2046-4053-1-41

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Sample size and power considerations in network meta-analysis

Abstract

Background

Findings

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Findings

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2012

An empirical study using permutation-based resampling in meta-regression

Access to regulatory data from the European Medicines Agency: the times they are a-changing

Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior

Debunking myths of protocol registration

Knowledge synthesis and the Canadian Institutes of Health Research

Antenatal magnesium individual participant data international collaboration: assessing the benefits for babies using the best level of evidence (AMICABLE)