Skip to main content
Top
Published in: Trials 1/2007

Open Access 01-12-2007 | Methodology

Application and investigation of a bound for outcome reporting bias

Authors: Paula R Williamson, Carrol Gamble

Published in: Trials | Issue 1/2007

Login to get access

Abstract

Background

Direct empirical evidence for the existence of outcome reporting bias is accumulating and this source of bias is recognised as a potential threat to the validity of meta-analysis of randomised clinical trials.

Methods

A method for calculating the maximum bias in a meta-analysis due to publication bias is adapted for the setting where within-study selective non-reporting of outcomes is suspected, and compared to the alternative approach of missing data imputation. The properties of both methods are investigated in realistic small sample situations.

Results

The results suggest that the adapted Copas and Jackson approach is the preferred method for reviewers to apply as an initial assessment of robustness to within-study selective non-reporting.

Conclusion

The Copas and Jackson approach is a useful method for systematic reviewers to apply to assess robustness to outcome reporting bias.
Appendix
Available only for authorised users
Literature
1.
go back to reference Dickersin K, Chans S, Chalmers T, Sacks H, Smith H: Publication bias and clinical trials. Controlled Clinical Trials. 1987, 8: 343-53. 10.1016/0197-2456(87)90155-3.CrossRefPubMed Dickersin K, Chans S, Chalmers T, Sacks H, Smith H: Publication bias and clinical trials. Controlled Clinical Trials. 1987, 8: 343-53. 10.1016/0197-2456(87)90155-3.CrossRefPubMed
2.
go back to reference Sutton AJ, Song F, Gilbody SM, Abrams KR: Modelling publication bias in meta-analysis: a review. Statistical Methods in Medical Research. 2000, 9: 421-445. 10.1191/096228000701555244.CrossRefPubMed Sutton AJ, Song F, Gilbody SM, Abrams KR: Modelling publication bias in meta-analysis: a review. Statistical Methods in Medical Research. 2000, 9: 421-445. 10.1191/096228000701555244.CrossRefPubMed
3.
go back to reference Copas J, Jackson D: A bound for publication bias based on the fraction of unpublished studies. Biometrics. 2004, 60: 146-153. 10.1111/j.0006-341X.2004.00161.x.CrossRefPubMed Copas J, Jackson D: A bound for publication bias based on the fraction of unpublished studies. Biometrics. 2004, 60: 146-153. 10.1111/j.0006-341X.2004.00161.x.CrossRefPubMed
4.
go back to reference Hutton JL, Williamson PR: Bias in meta-analysis due to outcome variable selection within studies. Applied Statistics. 2000, 49: 359-370. Hutton JL, Williamson PR: Bias in meta-analysis due to outcome variable selection within studies. Applied Statistics. 2000, 49: 359-370.
5.
go back to reference Williamson PR, Gamble C, Altman DG, Hutton JL: Within-study selective reporting bias in meta-analysis. Statistical Methods in Medical Research. 2005, 14: 515-524. 10.1191/0962280205sm415oa.CrossRefPubMed Williamson PR, Gamble C, Altman DG, Hutton JL: Within-study selective reporting bias in meta-analysis. Statistical Methods in Medical Research. 2005, 14: 515-524. 10.1191/0962280205sm415oa.CrossRefPubMed
6.
go back to reference Hahn S, Williamson PR, Hutton JL: Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to an LREC. Journal of Evaluation in Clinical Practice. 2002, 8: 353-360. 10.1046/j.1365-2753.2002.00314.x.CrossRefPubMed Hahn S, Williamson PR, Hutton JL: Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to an LREC. Journal of Evaluation in Clinical Practice. 2002, 8: 353-360. 10.1046/j.1365-2753.2002.00314.x.CrossRefPubMed
7.
go back to reference Chan A-W, Hrobjartsson A, Haahr M, Gotzsche PC, Altman DG: Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to publications. Journal of the American Medical Association. 2004, 291: 2457-2465. 10.1001/jama.291.20.2457.CrossRefPubMed Chan A-W, Hrobjartsson A, Haahr M, Gotzsche PC, Altman DG: Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to publications. Journal of the American Medical Association. 2004, 291: 2457-2465. 10.1001/jama.291.20.2457.CrossRefPubMed
8.
go back to reference Williamson PR, Gamble C: Identification and impact of outcome selection bias in meta-analysis. Statistics in Medicine. 2005, 24: 1547-1561. 10.1002/sim.2025.CrossRefPubMed Williamson PR, Gamble C: Identification and impact of outcome selection bias in meta-analysis. Statistics in Medicine. 2005, 24: 1547-1561. 10.1002/sim.2025.CrossRefPubMed
9.
go back to reference Ioannidis J, Trikalinos TA: Early extreme contradictory estimates may appear in published research: The Proteus phenomenon in molecular genetics research and randomised trials. Journal of Clinical Epidemiology. 2005, 58: 543-549. 10.1016/j.jclinepi.2004.10.019.CrossRefPubMed Ioannidis J, Trikalinos TA: Early extreme contradictory estimates may appear in published research: The Proteus phenomenon in molecular genetics research and randomised trials. Journal of Clinical Epidemiology. 2005, 58: 543-549. 10.1016/j.jclinepi.2004.10.019.CrossRefPubMed
10.
go back to reference Terrin N, Schmid CH, Lau J: In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. Journal of Clinical Epidemiology. 2005, 58: 894-901. 10.1016/j.jclinepi.2005.01.006.CrossRefPubMed Terrin N, Schmid CH, Lau J: In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. Journal of Clinical Epidemiology. 2005, 58: 894-901. 10.1016/j.jclinepi.2005.01.006.CrossRefPubMed
11.
go back to reference Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L: Comparison of two methods to detect publication bias in meta-analysis. Journal of the American Medical Association. 2006, 295: 676-680. 10.1001/jama.295.6.676.CrossRefPubMed Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L: Comparison of two methods to detect publication bias in meta-analysis. Journal of the American Medical Association. 2006, 295: 676-680. 10.1001/jama.295.6.676.CrossRefPubMed
12.
go back to reference Jackson D, Copas J, Sutton AJ: Modelling reporting bias: the operative mortality rate for ruptured abdominal aortic aneurysm repair. Journal of the Royal Statistical Society A. 2005, 168: 737-752. 10.1111/j.1467-985X.2005.00375.x.CrossRef Jackson D, Copas J, Sutton AJ: Modelling reporting bias: the operative mortality rate for ruptured abdominal aortic aneurysm repair. Journal of the Royal Statistical Society A. 2005, 168: 737-752. 10.1111/j.1467-985X.2005.00375.x.CrossRef
Metadata
Title
Application and investigation of a bound for outcome reporting bias
Authors
Paula R Williamson
Carrol Gamble
Publication date
01-12-2007
Publisher
BioMed Central
Published in
Trials / Issue 1/2007
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/1745-6215-8-9

Other articles of this Issue 1/2007

Trials 1/2007 Go to the issue