Published in:
Open Access
01-12-2008 | Research article
Undue reliance on I
2 in assessing heterogeneity may mislead
Authors:
Gerta Rücker, Guido Schwarzer, James R Carpenter, Martin Schumacher
Published in:
BMC Medical Research Methodology
|
Issue 1/2008
Login to get access
Abstract
Background
The heterogeneity statistic I
2, interpreted as the percentage of variability due to heterogeneity between studies rather than sampling error, depends on precision, that is, the size of the studies included.
Methods
Based on a real meta-analysis, we simulate artificially 'inflating' the sample size under the random effects model. For a given inflation factor
M = 1, 2, 3,... and for each trial
i, we create a
M-inflated trial by drawing a treatment effect estimate from the random effects model, using
/
M as within-trial sampling variance.
Results
As precision increases, while estimates of the heterogeneity variance τ
2 remain unchanged on average, estimates of I
2 increase rapidly to nearly 100%. A similar phenomenon is apparent in a sample of 157 meta-analyses.
Conclusion
When deciding whether or not to pool treatment estimates in a meta-analysis, the yard-stick should be the clinical relevance of any heterogeneity present. τ
2, rather than I
2, is the appropriate measure for this purpose.