Published in:
Open Access 01-12-2008 | Research article
Undue reliance on I 2 in assessing heterogeneity may mislead
Published in: BMC Medical Research Methodology | Issue 1/2008
Login to get accessAbstract
Background
The heterogeneity statistic I
2, interpreted as the percentage of variability due to heterogeneity between studies rather than sampling error, depends on precision, that is, the size of the studies included.
Methods
Based on a real meta-analysis, we simulate artificially 'inflating' the sample size under the random effects model. For a given inflation factor M = 1, 2, 3,... and for each trial i, we create a M-inflated trial by drawing a treatment effect estimate from the random effects model, using /M as within-trial sampling variance.
Results
As precision increases, while estimates of the heterogeneity variance τ
2 remain unchanged on average, estimates of I
2 increase rapidly to nearly 100%. A similar phenomenon is apparent in a sample of 157 meta-analyses.
Conclusion
When deciding whether or not to pool treatment estimates in a meta-analysis, the yard-stick should be the clinical relevance of any heterogeneity present. τ
2, rather than I
2, is the appropriate measure for this purpose.