Skip to main content
Top
Published in: Systematic Reviews 1/2018

Open Access 01-12-2018 | Research

Does information from ClinicalTrials.gov increase transparency and reduce bias? Results from a five-report case series

Authors: Gaelen P. Adam, Stacey Springs, Thomas Trikalinos, John W. Williams Jr, Jennifer L. Eaton, Megan Von Isenburg, Jennifer M. Gierisch, Lisa M. Wilson, Karen A. Robinson, Meera Viswanathan, Jennifer Cook Middleton, Valerie L. Forman-Hoffman, Elise Berliner, Robert M. Kaplan

Published in: Systematic Reviews | Issue 1/2018

Login to get access

Abstract

Background

We investigated whether information in ClinicalTrials.gov would impact the conclusions of five ongoing systematic reviews.

Method

We considered five reviews that included 495 studies total. Each review team conducted a search of ClinicalTrials.gov up to the date of the review’s last literature search, screened the records using the review’s eligibility criteria, extracted information, and assessed risk of bias and applicability. Each team then evaluated the impact of the evidence found in ClinicalTrials.gov on the conclusions in the review.

Results

Across the five reviews, the number of studies that had both a registry record and a publication varied widely, from none in one review to 43% of all studies identified in another. Among the studies with both a record and publication, there was also wide variability in the match between published outcomes and those listed in ClinicalTrials.gov. Of the 173 total ClinicalTrials.gov records identified across the five projects, between 11 and 43% did not have an associated publication. In the 14% of records that contained results, the new data provided in the ClinicalTrials.gov records did not change the results or conclusions of the reviews. Finally, a large number of published studies were not registered in ClinicalTrials.gov, but many of these were published before ClinicalTrials.gov’s inception date of 2000.

Conclusion

Improved prospective registration of trials and consistent reporting of results in ClinicalTrials.gov would help make ClinicalTrials.gov records more useful in finding unpublished information and identifying potential biases. In addition, consistent indexing in databases, such as MEDLINE, would allow for better matching of records and publications, leading to increased utility of these searches for systematic review projects.
Appendix
Available only for authorised users
Literature
1.
go back to reference Duval ST, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56:455–63.CrossRefPubMed Duval ST, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56:455–63.CrossRefPubMed
3.
go back to reference Harbord R, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med. 2006;25:3443–57.CrossRefPubMed Harbord R, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med. 2006;25:3443–57.CrossRefPubMed
4.
go back to reference Rucker G, Carpenter JR, Schwarzer G. Detecting and adjusting for small-study effects in meta-analysis. Biom J. 2011;53:351–68.CrossRefPubMed Rucker G, Carpenter JR, Schwarzer G. Detecting and adjusting for small-study effects in meta-analysis. Biom J. 2011;53:351–68.CrossRefPubMed
5.
go back to reference Copas J, Shi JQ. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics. 2000;1:247–62.CrossRefPubMed Copas J, Shi JQ. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics. 2000;1:247–62.CrossRefPubMed
6.
go back to reference Copas JB, Shi JQ. A sensitivity analysis for publication bias in systematic reviews. Stat Methods Med Res. 2001;10:251–65.CrossRefPubMed Copas JB, Shi JQ. A sensitivity analysis for publication bias in systematic reviews. Stat Methods Med Res. 2001;10:251–65.CrossRefPubMed
7.
go back to reference Hedges LVV, Jack L. Estimating effect size under publication bias: small sample properties and robustness of a random effects selection model. J Educ Behav Stat. 1996;21:299–332.CrossRef Hedges LVV, Jack L. Estimating effect size under publication bias: small sample properties and robustness of a random effects selection model. J Educ Behav Stat. 1996;21:299–332.CrossRef
8.
go back to reference Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, Williamson PR, Kirkham JJ. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11:e1001666.CrossRefPubMedPubMedCentral Dwan K, Altman DG, Clarke M, Gamble C, Higgins JP, Sterne JA, Williamson PR, Kirkham JJ. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11:e1001666.CrossRefPubMedPubMedCentral
11.
go back to reference De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJ, Schroeder TV, Sox HC, Van Der Weyden MB. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351:1250–1.CrossRefPubMed De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJ, Schroeder TV, Sox HC, Van Der Weyden MB. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351:1250–1.CrossRefPubMed
12.
go back to reference Zarin DA, Tse T, Sheehan J. The proposed rule for U.S. clinical trial registration and results submission. N Engl J Med. 2015;372:174–80.CrossRefPubMed Zarin DA, Tse T, Sheehan J. The proposed rule for U.S. clinical trial registration and results submission. N Engl J Med. 2015;372:174–80.CrossRefPubMed
13.
go back to reference Zarin DA, Tse T, Ross JS. Trial-results reporting and academic medical centers. N Engl J Med. 2015;372:2371–2.CrossRefPubMed Zarin DA, Tse T, Ross JS. Trial-results reporting and academic medical centers. N Engl J Med. 2015;372:2371–2.CrossRefPubMed
14.
go back to reference Wood AJ. Progress and deficiencies in the registration of clinical trials. N Engl J Med. 2009;360:824–30.CrossRefPubMed Wood AJ. Progress and deficiencies in the registration of clinical trials. N Engl J Med. 2009;360:824–30.CrossRefPubMed
15.
go back to reference Ioannidis JP, Trikalinos TA. Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. J Clin Epidemiol. 2005;58:543–9.CrossRefPubMed Ioannidis JP, Trikalinos TA. Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. J Clin Epidemiol. 2005;58:543–9.CrossRefPubMed
16.
go back to reference Roest AM, de Jonge P, Williams CD, de Vries YA, Schoevers RA, Turner EH. Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. JAMA Psychiatry. 2015;72:500–10.CrossRefPubMed Roest AM, de Jonge P, Williams CD, de Vries YA, Schoevers RA, Turner EH. Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. JAMA Psychiatry. 2015;72:500–10.CrossRefPubMed
17.
go back to reference Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963–71.CrossRefPubMed Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361:1963–71.CrossRefPubMed
18.
go back to reference Vedula SS, Li T, Dickersin K. Differences in reporting of analyses in internal company documents versus published trial reports: comparisons in industry-sponsored trials in off-label uses of gabapentin. PLoS Med. 2013;10:e1001378.CrossRefPubMedPubMedCentral Vedula SS, Li T, Dickersin K. Differences in reporting of analyses in internal company documents versus published trial reports: comparisons in industry-sponsored trials in off-label uses of gabapentin. PLoS Med. 2013;10:e1001378.CrossRefPubMedPubMedCentral
19.
go back to reference Agency for Healthcare Research and Quality (AHRQ). Methods guide for effectiveness and comparative effectiveness reviews. AHRQ publication no. 10(14)-EHC063-EF. Rockville: Agency for Healthcare Research and Quality; 2014. Agency for Healthcare Research and Quality (AHRQ). Methods guide for effectiveness and comparative effectiveness reviews. AHRQ publication no. 10(14)-EHC063-EF. Rockville: Agency for Healthcare Research and Quality; 2014.
20.
go back to reference Dy SM, Bennett WL, Sharma R, Zhang A, Waldfogel JM, Nesbit SA, Yeh HC, Chelladurai Y, Feldman D, Wilson LM, Robinson KA. AHRQ comparative effectiveness reviews. In: Preventing complications and treating symptoms of diabetic peripheral neuropathy. Rockville: Agency for Healthcare Research and Quality (US); 2017. Dy SM, Bennett WL, Sharma R, Zhang A, Waldfogel JM, Nesbit SA, Yeh HC, Chelladurai Y, Feldman D, Wilson LM, Robinson KA. AHRQ comparative effectiveness reviews. In: Preventing complications and treating symptoms of diabetic peripheral neuropathy. Rockville: Agency for Healthcare Research and Quality (US); 2017.
22.
go back to reference Balk EM, Lichtenstein AH. Omega-3 fatty acids and cardiovascular disease: summary of the 2016 Agency of Healthcare Research and Quality Evidence Review. Nutrients. 2017;9:865. Balk EM, Lichtenstein AH. Omega-3 fatty acids and cardiovascular disease: summary of the 2016 Agency of Healthcare Research and Quality Evidence Review. Nutrients. 2017;9:865.
23.
go back to reference Steele D, Adam GP, Di M, Halladay C, Pan I, Coppersmith N, Balk EM, Trikalinos TA. AHRQ comparative effectiveness reviews. In: Tympanostomy tubes in children with otitis media. Rockville: Agency for Healthcare Research and Quality (US); 2017.CrossRef Steele D, Adam GP, Di M, Halladay C, Pan I, Coppersmith N, Balk EM, Trikalinos TA. AHRQ comparative effectiveness reviews. In: Tympanostomy tubes in children with otitis media. Rockville: Agency for Healthcare Research and Quality (US); 2017.CrossRef
24.
go back to reference Forman-Hoffman VL, Middleton JC, McKeeman JL, Stambaugh LF, Christian RB, Gaynes BN, Kane HL, Kahwati LC, Lohr KN, Viswanathan M. AHRQ comparative effectiveness reviews. In: Strategies to improve mental health care for children and adolescents. Rockville: Agency for Healthcare Research and Quality (US); 2016. Forman-Hoffman VL, Middleton JC, McKeeman JL, Stambaugh LF, Christian RB, Gaynes BN, Kane HL, Kahwati LC, Lohr KN, Viswanathan M. AHRQ comparative effectiveness reviews. In: Strategies to improve mental health care for children and adolescents. Rockville: Agency for Healthcare Research and Quality (US); 2016.
25.
go back to reference Adam GP, Springs S, Langberg V, Halliday C, Trikalinos TA. Supplemental project to assess the transparency of reporting requirements: tympanostomy tubes in children with otitis media. Rockville: Agency for Healthcare Research and Quality; 2017. Adam GP, Springs S, Langberg V, Halliday C, Trikalinos TA. Supplemental project to assess the transparency of reporting requirements: tympanostomy tubes in children with otitis media. Rockville: Agency for Healthcare Research and Quality; 2017.
26.
go back to reference Springs S, Adam GP, Langberg V, Halliday C, Trikalinos TA. Supplemental project to assess the transparency of reporting requirements: omega-3 fatty acids and cardiovascular disease. Rockville: Agency for Healthcare Research and Quality; 2017. Springs S, Adam GP, Langberg V, Halliday C, Trikalinos TA. Supplemental project to assess the transparency of reporting requirements: omega-3 fatty acids and cardiovascular disease. Rockville: Agency for Healthcare Research and Quality; 2017.
27.
go back to reference Viswanathan M, Middleton JC, Forman-Hoffman VL. AHRQ methods for effective health care. In: Supplemental project to assess the transparency of reporting for strategies to improve mental health care for children and adolescents. Rockville: Agency for Healthcare Research and Quality (US); 2017. Viswanathan M, Middleton JC, Forman-Hoffman VL. AHRQ methods for effective health care. In: Supplemental project to assess the transparency of reporting for strategies to improve mental health care for children and adolescents. Rockville: Agency for Healthcare Research and Quality (US); 2017.
28.
go back to reference Williams JW Jr, Eaton JL, Gierisch JM, Masilamani V, von Isenburg M, Chobot M. Supplemental project to assess the transparency of reporting for trials evaluating treatment for infertility. Rockville: Agency for Healthcare Research and Quality; 2017.CrossRef Williams JW Jr, Eaton JL, Gierisch JM, Masilamani V, von Isenburg M, Chobot M. Supplemental project to assess the transparency of reporting for trials evaluating treatment for infertility. Rockville: Agency for Healthcare Research and Quality; 2017.CrossRef
29.
go back to reference Wilson LM, Sharma R, Dy SM, Waldfogel JM, Robinson KA. Searching ClinicalTrials.gov did not change the conclusions of a systematic review. J Clin Epidemiol. 2017;90:127–35. Wilson LM, Sharma R, Dy SM, Waldfogel JM, Robinson KA. Searching ClinicalTrials.gov did not change the conclusions of a systematic review. J Clin Epidemiol. 2017;90:127–35.
30.
go back to reference Baudard M, Yavchitz A, Ravaud P, Perrodeau E, Boutron I. Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses. BMJ. 2017;356:j448.CrossRefPubMedPubMedCentral Baudard M, Yavchitz A, Ravaud P, Perrodeau E, Boutron I. Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses. BMJ. 2017;356:j448.CrossRefPubMedPubMedCentral
31.
go back to reference Tang E, Ravaud P, Riveros C, Perrodeau E, Dechartres A. Comparison of serious adverse events posted at ClinicalTrials.gov and published in corresponding journal articles. BMC Med. 2015;13:189.CrossRefPubMedPubMedCentral Tang E, Ravaud P, Riveros C, Perrodeau E, Dechartres A. Comparison of serious adverse events posted at ClinicalTrials.gov and published in corresponding journal articles. BMC Med. 2015;13:189.CrossRefPubMedPubMedCentral
32.
go back to reference Hartung DM, Zarin DA, Guise JM, McDonagh M, Paynter R, Helfand M. Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications. Ann Intern Med. 2014;160:477–83.CrossRefPubMedPubMedCentral Hartung DM, Zarin DA, Guise JM, McDonagh M, Paynter R, Helfand M. Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications. Ann Intern Med. 2014;160:477–83.CrossRefPubMedPubMedCentral
33.
Metadata
Title
Does information from ClinicalTrials.gov increase transparency and reduce bias? Results from a five-report case series
Authors
Gaelen P. Adam
Stacey Springs
Thomas Trikalinos
John W. Williams Jr
Jennifer L. Eaton
Megan Von Isenburg
Jennifer M. Gierisch
Lisa M. Wilson
Karen A. Robinson
Meera Viswanathan
Jennifer Cook Middleton
Valerie L. Forman-Hoffman
Elise Berliner
Robert M. Kaplan
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2018
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/s13643-018-0726-5

Other articles of this Issue 1/2018

Systematic Reviews 1/2018 Go to the issue