Top

Trials

Published in:

Open Access 01-12-2019 | Quetiapine | Research

Opportunities for selective reporting of harms in randomized clinical trials: Selection criteria for non-systematic adverse events

Authors: Evan Mayo-Wilson, Nicole Fusco, Hwanhee Hong, Tianjing Li, Joseph K. Canner, Kay Dickersin

Published in: Trials | Issue 1/2019

Abstract

Background

Adverse events (AEs) in clinical trials may be reported in multiple sources. Different methods for reporting adverse events across trials or across sources for a single trial may produce inconsistent information about the adverse events associated with interventions.

Methods

We compared the methods authors use to decide which AEs to include in a particular source (i.e., “selection criteria”), including the number of different types of AEs reported (i.e., rather than the number of events). We compared sources (e.g., journal articles, clinical study reports (CSRs)) of trials for two drug-indications—gabapentin for neuropathic pain and quetiapine for bipolar depression. Electronic searches were completed in 2015. We identified selection criteria and assessed how criteria affected AE reporting.

Results

We identified 21 gabapentin and 7 quetiapine trials. We found 6 gabapentin CSRs and 2 quetiapine CSRs, all written by drug manufacturers. All CSRs reported all AEs without applying selection criteria; by comparison, no other source reported all AEs, and 15/68 (22%) gabapentin sources and 19/48 (40%) quetiapine sources reported using selection criteria. Selection criteria greatly affected the number of AEs reported. For example, 67/316 (21%) AEs in one quetiapine trial met the criterion “occurring in ≥2% of participants in any treatment group,” while only 5/316 (2%) AEs met the criterion “occurring in ≥10% of quetiapine-treated patients and twice as frequent in the quetiapine group as the placebo group.”

Conclusions

Selection criteria for reporting AEs vary across trials and across sources for individual trials. If investigators do not pre-specify selection criteria, they might “cherry-pick” AEs based on results. Even if investigators pre-specify selection criteria, selective reporting will produce biased meta-analyses and clinical practice guidelines. Data about all AEs identified in clinical trials should be publicly available; however, sharing data will not solve all the problems identified in this study.

Available only for authorised users

Department of Health and Human Services. 42 CFR Part 11. Clinical trials registration and results information submission: Final rule. 2016. https://www.federalregister.gov/documents/2016/09/21/2016-22129/clinical-trials-registration-and-results-information-submission. Accessed 12 May 2018.

Zarin DA, Tse T, Williams RJ, et al. Trial reporting in ClinicalTrials. gov—the final rule. N Engl J Med. 2016;375(20):1998–2004. https://doi.org/10.1056/NEJMsr1611785.CrossRefPubMedPubMedCentral

Department of Health and Human Services. CFR Title 21, Section 312. Investigational new drug application, 2010.

European Medicines Agency. Guideline on good pharmacovigilance practices (GVP): Annex I - Definitions (Rev 4), 2017. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2013/05/WC500143294.pdf. Accessed 12 May 2018.

Department of Health and Human Services. CFR Title 21, Section 312.32. IND safety reporting. 2010. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRsearch.cfm?CFRPart=312. Accessed 12 May 2018.

Golder S, Loke YK, Wright K, et al. Most systematic reviews of adverse effects did not include unpublished data. J Clin Epidemiol. 2016;77:125–33. https://doi.org/10.1016/S0889-8588(05)70309-9.CrossRefPubMed

Parsons R, Golder S, Watt I. More than one-third of systematic reviews did not fully report the adverse events outcome. J Clin Epidemiol. 2019;108:95–101. https://doi.org/10.1016/j.jclinepi.2018.12.007.CrossRefPubMed

Bonini S, Eichler H-G, Wathion N, et al. Transparency and the European Medicines Agency—sharing of clinical trial data. N Engl J Med. 2014;371(26):2452–5.CrossRefPubMed

Doshi P, Jefferson T. The first 2 years of the European Medicines Agency's policy on access to documents: secret no longer. JAMA Intern Med. 2013;173(5):380–2.CrossRefPubMed

10.

U.S. Food and Drug Administration. FDA Commissioner Scott Gottlieb, M.D., on new steps FDA is taking to enhance transparency of clinical trial information to support innovation and scientific inquiry related to new drugs [press release]. 2018.

11.

Bierer BE, Li R, Barnes M, et al. A global, neutral platform for sharing trial data. N Engl J Med. 2016;374(25):2411–3.CrossRefPubMed

12.

Krumholz HM, Waldstreicher J. The Yale Open Data Access (YODA) project—a mechanism for data sharing. N Engl J Med. 2016;375(5):403–5.CrossRefPubMed

13.

Ross JS, Waldstreicher J, Bamford S, et al. Overview and experience of the YODA project with clinical trial data sharing after 5 years. Sci Data. 2018;5:180268.CrossRefPubMedPubMedCentral

14.

Dickersin K, Min Y-I, Meinert CL. Factors influencing publication of research results: follow-up of applications submitted to two institutional review boards. JAMA. 1992;267(3):374–8. https://doi.org/10.1001/jama.1992.03480030052036.CrossRefPubMed

15.

Dwan K, Gamble C, Williamson PR, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One. 2013;8(7):e66844. https://doi.org/10.1371/journal.pone.0066844.CrossRefPubMedPubMedCentral

16.

Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330(7494):753–6. https://doi.org/10.1136/bmj.38356.424606.8F.CrossRefPubMedPubMedCentral

17.

Chan AW, Hrobjartsson A, Haahr MT, et al. Empirical evidence for selective reporting of outcomes in randomized trials - comparison of protocols to published articles. JAMA. 2004;291(20):2457–65. https://doi.org/10.1001/jama.291.20.2457.CrossRefPubMed

18.

Chan AW, Krieza-Jeric K, Schmid I, et al. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. Can Med Assoc J. 2004;171(7):735–40. https://doi.org/10.1503/cmaj.1041086.CrossRef

19.

Dwan K, Altman DG, Clarke M, et al. Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med. 2014;11(6):e1001666. https://doi.org/10.1371/journal.pmed.1001666.CrossRefPubMedPubMedCentral

20.

Mayo-Wilson E, Fusco N, Li T, et al. Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. J Clin Epidemiol. 2017;86:39–50. https://doi.org/10.1016/j.jclinepi.2017.05.007.CrossRefPubMed

21.

Vedula SS, Bero L, Scherer RW, et al. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361(20):1963–71. https://doi.org/10.1056/NEJMsa0906126.CrossRefPubMed

22.

Mayo-Wilson E, Fusco N, Li T, et al. Harms are assessed inconsistently and reported inadequately Part 2: Non-systematic adverse events. J Clin Epidemiol. 2019;113:11–9. https://doi.org/10.1016/j.jclinepi.2019.04.020.CrossRefPubMed

23.

Mayo-Wilson E, Fusco N, Li T, et al. Harms are assessed inconsistently and reported inadequately Part 1: Systematic adverse events. J Clin Epidemiol. 2019;113:20–7. https://doi.org/10.1016/j.jclinepi.2019.04.022.CrossRefPubMed

24.

Mayo-Wilson E, Li T, Fusco N, et al. Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy. J Clin Epidemiol. 2017;91:95–110. https://doi.org/10.1016/j.jclinepi.2017.07.014.CrossRefPubMed

25.

Golder S, Loke YK, Wright K, et al. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Med. 2016;13(9):e1002127. https://doi.org/10.1371/journal.pmed.1002127.CrossRefPubMedPubMedCentral

26.

Halfpenny NJA, Quigley JM, Thompson JC, et al. Value and usability of unpublished data sources for systematic reviews and network meta-analyses. Evid Based Med. 2016;21(6):208–13. https://doi.org/10.1136/ebmed-2016-110494.CrossRefPubMed

27.

Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344:d7202. https://doi.org/10.1136/bmj.d7202.CrossRefPubMed

28.

Hartung DM, Zarin DA, Guise J-M, et al. Reporting discrepancies between the ClinicalTrials. gov results database and peer-reviewed publications. Ann Intern Med. 2014;160(7):477–83. https://doi.org/10.7326/M13-0480.CrossRefPubMedPubMedCentral

29.

Jefferson T, Jones MA, Doshi P, et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev. 2012;1:CD008965. https://doi.org/10.1002/14651858.CD008965.pub4.CrossRefPubMed

30.

Le Noury J, Nardo JM, Healy D, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ. 2015;351:h4320. https://doi.org/10.1136/bmj.h4320.CrossRefPubMedPubMedCentral

31.

Riveros C, Dechartres A, Perrodeau E, et al. Timing and completeness of trial results posted at ClinicalTrials. gov and published in journals. PLoS Med. 2013;10(12):e1001566. https://doi.org/10.1371/journal.pmed.1001566.CrossRefPubMedPubMedCentral

32.

Rodgers MA, Brown JV, Heirs MK, et al. Reporting of industry funded study outcome data: comparison of confidential and published data on the safety and effectiveness of rhBMP-2 for spinal fusion. BMJ. 2013;346:f3981. https://doi.org/10.1136/bmj.f3981.CrossRefPubMedPubMedCentral

33.

International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH harmonised tripartite guideline: clinical safety data management: definitions and standards for expedited reporting E2A. 1994. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E2A/Step4/E2A_Guideline.pdf; Accessed 12 May 2018.

34.

Zarin DA, Tse T, Williams RJ, et al. The ClinicalTrials.gov results database--update and key issues. N Engl J Med. 2011;364(9):852–60. https://doi.org/10.1056/NEJMsa1012065.CrossRefPubMedPubMedCentral

35.

International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH harmonised tripartite guideline: structure and content of clinical study reports E3. 1995. http://www.ich.org/products/guidelines/efficacy/efficacy-single/article/structure-and-content-of-clinical-study-reports.html. Accessed 12 May 2018.

36.

Lineberry N, Berlin JA, Mansi B, et al. Recommendations to improve adverse event reporting in clinical trial publications: a joint pharmaceutical industry/journal editor perspective. BMJ. 2016;355:i5078. https://doi.org/10.1136/bmj.i5078.CrossRefPubMed

37.

Honvo G, Bannuru RR, Bruyère O, et al. Recommendations for the reporting of harms in manuscripts on clinical trials assessing osteoarthritis drugs: a consensus statement from the European Society for Clinical and Economic Aspects of Osteoporosis, Osteoarthritis and Musculoskeletal Diseases (ESCEO). Drugs Aging. 2019;36(1):145–59.CrossRefPubMedPubMedCentral

38.

Ioannidis JP, Evans SJ, Gøtzsche PC, et al. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004;141(10):781–8. https://doi.org/10.7326/0003-4819-141-10-200411160-00009.CrossRefPubMed

39.

Mayo-Wilson E, Hutfless S, Li T, et al. Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol for a systematic review. Syst Rev. 2015;4:143. https://doi.org/10.1186/s13643-015-0134-z (Correction: 2018; 7:48).CrossRefPubMedPubMedCentral

40.

Mayo-Wilson E, Doshi P, Dickersin K. Are manufacturers sharing data as promised? BMJ. 2015;351:h4169. https://doi.org/10.1136/bmj.h4169.CrossRefPubMed

41.

Stata Statistical Software: Release 14 [program]. College Station; 2015.

42.

AstraZeneca. A multicenter, double-blind, randomized, placebo-controlled, double- dummy trial of the use of quetiapine fumarate (SEROQUEL [Registered]) in the treatment of patients with bipolar depression. Study code: 5077US/0049, 2005.

43.

Crowe BJ, Xia HA, Berlin JA, et al. Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team. Clin Trials. 2009;6(5):430–40.CrossRefPubMed

44.

Chalmers I. Proposal to outlaw the term “negative trial”. BMJ. 1985;290(6473):1002.CrossRef

45.

Hutton J, Williamson PR. Bias in meta-analysis due to outcome variable selection within studies. J Roy Stat Soc C App. 2000;49(3):359–70. https://doi.org/10.1111/1467-9876.00197.CrossRef

46.

Kirkham JJ, Dwan KM, Altman DG, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365. https://doi.org/10.1136/bmj.c365.CrossRefPubMed

47.

Williamson PR, Gamble C. Identification and impact of outcome selection bias in meta-analysis. Stat Med. 2005;24(10):1547–61. https://doi.org/10.1002/sim.2025.CrossRefPubMed

48.

Williamson PR, Gamble C, Altman DG, et al. Outcome selection bias in meta-analysis. Stat Methods Med Res. 2005;14(5):515–24. https://doi.org/10.1191/0962280205sm415oa [published Online First: 2005/10/27].CrossRefPubMed

49.

110th Congress of the United States of America. Food and Drug Administration Amendments Act, Section 801. 2007.

50.

Mayo-Wilson E, Li T, Fusco N, et al. Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study). Res Synth Methods. 2018;9(1):2–12. https://doi.org/10.1002/jrsm.1277.CrossRefPubMed

51.

Mayo-Wilson E, Golozar A, Cowley T, et al. Methods to identify and prioritize patient-centered outcomes for use in comparative effectiveness research. Pilot Feasibility Stud. 2018;4(95). https://doi.org/10.1186/s40814-018-0284-6.

52.

Prinsen CA, Vohra S, Rose MR, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set’. Trials. 2014;15(1):247.CrossRefPubMedPubMedCentral

53.

COMET Initiative. http://www.comet-initiative.org/. Accessed 12 May 2018.

54.

Dickersin K, Mayo-Wilson E. Standards for design and measurement would make clinical research reproducible and usable. P Natl Acad Sci USA. 2018;115(11):2590–4. https://doi.org/10.1073/pnas.1708273114.CrossRef

Title: Opportunities for selective reporting of harms in randomized clinical trials: Selection criteria for non-systematic adverse events
Authors: Evan Mayo-Wilson
Nicole Fusco
Hwanhee Hong
Tianjing Li
Joseph K. Canner
Kay Dickersin
Publication date: 01-12-2019
Publisher: BioMed Central
Keywords: Quetiapine
Gabapentin
Published in: Trials / Issue 1/2019
Electronic ISSN: 1745-6215
DOI: https://doi.org/10.1186/s13063-019-3581-3

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Opportunities for selective reporting of harms in randomized clinical trials: Selection criteria for non-systematic adverse events

Abstract

Background

Methods

Results

Conclusions

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2019

Correction to: Use of daily electronic patient-reported outcome (PRO) diaries in randomized controlled trials for rheumatoid arthritis: rationale and implementation

Effects of home-based prescribed pulmonary exercise by patients with chronic obstructive pulmonary disease: study protocol for a randomized controlled trial

The impact of surgeon and patient treatment preferences in an orthopaedic trauma surgery trial

The TREATT Trial (TRial to EvaluAte Tranexamic acid therapy in Thrombocytopenia): safety and efficacy of tranexamic acid in patients with haematological malignancies with severe thrombocytopenia: study protocol for a double-blind randomised controlled trial

Determining the effects of exercise after smoking cessation therapy completion on continuous abstinence from smoking: Japanese study protocol

Executive function training in chronic traumatic brain injury patients: study protocol