Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Trials
Published in: Trials 1/2018

Open Access 01-12-2018 | Methodology

Outcome pre-specification requires sufficient detail to guard against outcome switching in clinical trials: a case study

Authors: Brennan C. Kahan, Vipul Jairath

Published in: Trials | Issue 1/2018

Login to get access

Abstract

Background

Pre-specification of outcomes is an important tool to guard against outcome switching in clinical trials. However, if the outcome is not sufficiently clearly defined, then different definitions could be applied and analysed, with only the most favourable result reported.

Methods

In order to assess the impact that differing outcome definitions could have on treatment effect estimates, we re-analysed data from TRIGGER, a cluster randomised trial comparing two red blood cell transfusion strategies for patients with acute upper gastrointestinal bleeding. We varied several aspects of the definition of further bleeding: (1) the criteria for what constitutes a further bleeding episode; (2) how further bleeding is assessed; and (3) the time-point at which further bleeding is measured.

Results

There were marked discrepancies in the estimated odds ratios (OR) (range 0.23–0.94) and corresponding P values (range < 0.001–0.89) between different outcome definitions. At the extremes, differing outcome definitions led to markedly different conclusions; one definition led to very little evidence of a treatment effect (OR = 0.94, 95% confidence interval [CI] = 0.37–2.40, P = 0.89), while another led to very strong evidence of a treatment effect (OR = 0.23, 95% CI = 0.11–0.50, P < 0.001).

Conclusions

Outcomes should be pre-specified in sufficient detail to avoid differing definitions being analysed and only the most favourable result being reported.

Trial registration

Clinical Trials.gov, NCT02105532. Registered on 7 April 2014.
Literature
1.
Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330(7494):753.CrossRefPubMedPubMedCentral
2.
Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–65.CrossRefPubMed
3.
Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.CrossRefPubMedPubMedCentral
4.
Contopoulos-Ioannidis DG, Alexiou GA, Gouvias TC, Ioannidis JP. An empirical evaluation of multifarious outcomes in pharmacogenetics: beta-2 adrenoceptor gene polymorphisms in asthma treatment. Pharmacogenet Genomics. 2006;16(10):705–11.CrossRefPubMed
5.
Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3(8):e3081.CrossRefPubMedPubMedCentral
6.
7.
Hahn S, Williamson PR, Hutton JL. Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. J Eval Clin Pract. 2002;8(3):353–9.CrossRefPubMed
8.
Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med. 2008;5(11):e217. discussion e217CrossRefPubMedPubMedCentral
9.
Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361(20):1963–71.CrossRefPubMed
10.
Williamson PR, Gamble C, Altman DG, Hutton JL. Outcome selection bias in meta-analysis. Stat Methods Med Res. 2005;14(5):515–24.CrossRefPubMed
11.
Page MJ, Forbes A, Chau M, Green SE, McKenzie JE. Investigation of bias in meta-analyses due to selective inclusion of trial effect estimates: empirical study. BMJ Open. 2016;6(4):e011863.CrossRefPubMedPubMedCentral
12.
Mayo-Wilson E, Li T, Fusco N, Bertizzolo L, Canner JK, Cowley T, et al. Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy. J Clin Epidemiol. 2017;91:95–110.CrossRefPubMed
13.
Ramagopalan S, Skingsley AP, Handunnetthi L, Klingel M, Magnus D, Pakpoor J, et al. Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study. F1000Res. 2014;3:77.PubMedPubMedCentral
14.
15.
Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K, investigators M. Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. J Clin Epidemiol. 2017;86:39–50.CrossRefPubMed
16.
Jairath V, Kahan BC, Gray A, Dore CJ, Mora A, Dyer C, et al. Restrictive vs liberal blood transfusion for acute upper gastrointestinal bleeding: rationale and protocol for a cluster randomized feasibility trial. Transfus Med Rev. 2013;27(3):146–53.CrossRefPubMedPubMedCentral
17.
Jairath V, Kahan BC, Gray A, Dore CJ, Mora A, James MW, et al. Restrictive versus liberal blood transfusion for acute upper gastrointestinal bleeding (TRIGGER): a pragmatic, open-label, cluster randomised feasibility trial. Lancet. 2015;386(9989):137–44.CrossRefPubMed
18.
Kahan BC, Jairath V, Murphy MF, Dore CJ. Update on the transfusion in gastrointestinal bleeding (TRIGGER) trial: statistical analysis plan for a cluster-randomised feasibility trial. Trials. 2013;14:206.CrossRefPubMedPubMedCentral
19.
Laine L, Spiegel B, Rostom A, Moayyedi P, Kuipers EJ, Bardou M, et al. Methodology for randomized trials of patients with nonvariceal upper gastrointestinal bleeding: recommendations from an international consensus conference. Am J Gastroenterol. 2010;105(3):540–50.CrossRefPubMed
20.
Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73(1):13–22.CrossRef
21.
Kahan BC, Jairath V, Dore CJ, Morris TP. The risks and rewards of covariate adjustment in randomized trials: an assessment of 12 outcomes from 8 studies. Trials. 2014;15:139.CrossRefPubMedPubMedCentral
22.
Leyrat C, Caille A, Donner A, Giraudeau B. Propensity scores used for analysis of cluster randomized trials with selection bias: a simulation study. Stat Med. 2013;32(19):3357–72.CrossRefPubMed
Metadata
Title
Outcome pre-specification requires sufficient detail to guard against outcome switching in clinical trials: a case study
Authors
Brennan C. Kahan
Vipul Jairath
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Trials / Issue 1/2018
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/s13063-018-2654-z

Other articles of this Issue 1/2018

Trials 1/2018 Go to the issue

Study protocol

A multicenter, randomized controlled comparison of three renutrition strategies for the management of moderate acute malnutrition among children aged from 6 to 24 months (the MALINEA project)

Review

N-of-1 trials in the clinical care of patients in developing countries: a systematic review

Study protocol

An eHealth program versus a standard care supervised health program and associated health outcomes in individuals with mobility disability: study protocol for a randomized controlled trial

Study protocol

The central mechanism of acupuncture treatment with a long-lasting effect for functional dyspepsia: study protocol for a randomized controlled trial

Study protocol

Implementation of evidence-based practice for benign paroxysmal positional vertigo: DIZZTINCT– A study protocol for an exploratory stepped-wedge randomized trial

Research

A novel integrated dressing to secure peripheral intravenous catheters in an adult acute hospital: a pilot randomised controlled trial