Validity of a Cochrane Systematic Review and meta-analysis for determining the safety of vitamin E

The Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study (ATBC), a factorial design study of Finnish smokers was a pivotal paper in the Cochrane Review owing to the high mortality rate observed which resulted in a substantial weighting (42.6%) in the meta-analysis. The Cochrane meta-analysis used a 3 cell analytical method comparing all vitamin E cells (vitamin E alone plus vitamin E + β-carotene) to the placebo only cell. This had the unfortunate effect of incorrectly inflating the mortality risk attributed to vitamin E by not balancing the contribution to mortality of the β-carotene intervention. Re-analysis of the ATBC trial using data derived from the more generally accepted ‘inside the table’ (2 cell – vitamin E versus placebo) or ‘at the margins’ (4 cell – all vitamin E versus all non-vitamin E) analytical methods demonstrates a statistically non-significant result.

The data from the ATBC study has been given in 5 datasets (the trial alone and four extended post-trial follow-up time periods). Using the 3 cell analysis method only the 6 and 8-year (used in the meta-analysis) follow-up periods were statistically significant. Using the 2 or 4 cell method the outcome remains non-significant over all time periods.

The impartiality of excluding trials with zero mortality is also examined and questioned.

This paper raises concerns overall as to the appropriateness of datasets chosen, the validity of methods and generalisability of results when using meta-analysis as a tool to determine safety. Issues raised in this paper are not unique to the Cochrane study in question. Until we have new tools, there may be a need to rely on conventional narrative systematic literature synthesis in the assessment of safety or contain our results to specific sub-populations where more conclusive results can be determined.