Skip to main content
Top
Published in: Trials 1/2014

Open Access 01-12-2014 | Research

Reducing bias in open-label trials where blinded outcome assessment is not feasible: strategies from two randomised trials

Authors: Brennan C Kahan, Suzie Cro, Caroline J Doré, Daniel J Bratton, Sunita Rehal, Nick A Maskell, Najib Rahman, Vipul Jairath

Published in: Trials | Issue 1/2014

Login to get access

Abstract

Background

Blinded outcome assessment is recommended in open-label trials to reduce bias, however it is not always feasible. It is therefore important to find other means of reducing bias in these scenarios.

Methods

We describe two randomised trials where blinded outcome assessment was not possible, and discuss the strategies used to reduce the possibility of bias.

Results

TRIGGER was an open-label cluster randomised trial whose primary outcome was further bleeding. Because of the cluster randomisation, all researchers in a hospital were aware of treatment allocation and so could not perform a blinded assessment. A blinded adjudication committee was also not feasible as it was impossible to compile relevant information to send to the committee in a blinded manner. Therefore, the definition of further bleeding was modified to exclude subjective aspects (such as whether symptoms like vomiting blood were severe enough to indicate the outcome had been met), leaving only objective aspects (the presence versus absence of active bleeding in the upper gastrointestinal tract confirmed by an internal examination).
TAPPS was an open-label trial whose primary outcome was whether the patient was referred for a pleural drainage procedure. Allowing a blinded assessor to decide whether to refer the patient for a procedure was not feasible as many clinicians may be reluctant to enrol patients into the trial if they cannot be involved in their care during follow-up. Assessment by an adjudication committee was not possible, as the outcome either occurred or did not. Therefore, the decision pathway for procedure referral was modified. If a chest x-ray indicated that more than a third of the pleural space filled with fluid, the patient could be referred for a procedure; otherwise, the unblinded clinician was required to reach a consensus on referral with a blinded assessor. This process allowed the unblinded clinician to be involved in the patient’s care, while reducing the potential for bias.

Conclusions

When blinded outcome assessment is not possible, it may be useful to modify the outcome definition or method of assessment to reduce the risk of bias.

Trial registration

TRIGGER: ISRCTN85757829. Registered 26 July 2012.
TAPPS: ISRCTN47845793. Registered 28 May 2012.
Literature
1.
go back to reference Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hrobjartsson A, Schulz KF, Parulekar WR, Krleza-Jeric K, Laupacis A, Moher D: SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013, 346: e7586-10.1136/bmj.e7586.CrossRefPubMedPubMedCentral Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hrobjartsson A, Schulz KF, Parulekar WR, Krleza-Jeric K, Laupacis A, Moher D: SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013, 346: e7586-10.1136/bmj.e7586.CrossRefPubMedPubMedCentral
2.
go back to reference Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995, 273 (5): 408-412. 10.1001/jama.1995.03520290060030.CrossRefPubMed Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995, 273 (5): 408-412. 10.1001/jama.1995.03520290060030.CrossRefPubMed
3.
go back to reference Schulz KF, Grimes DA: Blinding in randomised trials: hiding who got what. Lancet. 2002, 359 (9307): 696-700. 10.1016/S0140-6736(02)07816-9.CrossRefPubMed Schulz KF, Grimes DA: Blinding in randomised trials: hiding who got what. Lancet. 2002, 359 (9307): 696-700. 10.1016/S0140-6736(02)07816-9.CrossRefPubMed
4.
go back to reference Boutron I, Tubach F, Giraudeau B, Ravaud P: Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol. 2004, 57 (6): 543-550. 10.1016/j.jclinepi.2003.12.010.CrossRefPubMed Boutron I, Tubach F, Giraudeau B, Ravaud P: Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol. 2004, 57 (6): 543-550. 10.1016/j.jclinepi.2003.12.010.CrossRefPubMed
5.
go back to reference Wright S, Duncombe P, Altman DG: Assessment of blinding to treatment allocation in studies of a cannabis-based medicine (Sativex (R)) in people with multiple sclerosis: a new approach. Trials. 2012, 13: 189-10.1186/1745-6215-13-189.CrossRefPubMedPubMedCentral Wright S, Duncombe P, Altman DG: Assessment of blinding to treatment allocation in studies of a cannabis-based medicine (Sativex (R)) in people with multiple sclerosis: a new approach. Trials. 2012, 13: 189-10.1186/1745-6215-13-189.CrossRefPubMedPubMedCentral
6.
go back to reference Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S: Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012, 344: e1119-10.1136/bmj.e1119.CrossRefPubMed Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S: Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012, 344: e1119-10.1136/bmj.e1119.CrossRefPubMed
7.
go back to reference Linde K, Witt CM, Streng A, Weidenhammer W, Wagenpfeil S, Brinkhaus B, Willich SN, Melchart D: The impact of patient expectations on outcomes in four randomized controlled trials of acupuncture in patients with chronic pain. Pain. 2007, 128 (3): 264-271. 10.1016/j.pain.2006.12.006.CrossRefPubMed Linde K, Witt CM, Streng A, Weidenhammer W, Wagenpfeil S, Brinkhaus B, Willich SN, Melchart D: The impact of patient expectations on outcomes in four randomized controlled trials of acupuncture in patients with chronic pain. Pain. 2007, 128 (3): 264-271. 10.1016/j.pain.2006.12.006.CrossRefPubMed
8.
go back to reference Noseworthy JH, Ebers GC, Vandervoort MK, Farquhar RE, Yetisir E, Roberts R: The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial. Neurology. 1994, 44 (1): 16-20. 10.1212/WNL.44.1.16.CrossRefPubMed Noseworthy JH, Ebers GC, Vandervoort MK, Farquhar RE, Yetisir E, Roberts R: The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial. Neurology. 1994, 44 (1): 16-20. 10.1212/WNL.44.1.16.CrossRefPubMed
9.
go back to reference Poolman RW, Struijs PA, Krips R, Sierevelt IN, Marti RK, Farrokhyar F, Bhandari M: Reporting of outcomes in orthopaedic randomized trials: does blinding of outcome assessors matter?. J Bone Joint Surg Am. 2007, 89 (3): 550-558. 10.2106/JBJS.F.00683.CrossRefPubMed Poolman RW, Struijs PA, Krips R, Sierevelt IN, Marti RK, Farrokhyar F, Bhandari M: Reporting of outcomes in orthopaedic randomized trials: does blinding of outcome assessors matter?. J Bone Joint Surg Am. 2007, 89 (3): 550-558. 10.2106/JBJS.F.00683.CrossRefPubMed
10.
go back to reference Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008, 336 (7644): 601-605. 10.1136/bmj.39465.451748.AD.CrossRefPubMedPubMedCentral Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008, 336 (7644): 601-605. 10.1136/bmj.39465.451748.AD.CrossRefPubMedPubMedCentral
11.
go back to reference Savovic J, Jones HE, Altman DG, Harris RJ, Juni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JP, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JA: Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012, 157 (6): 429-438. 10.7326/0003-4819-157-6-201209180-00537.CrossRefPubMed Savovic J, Jones HE, Altman DG, Harris RJ, Juni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JP, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JA: Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012, 157 (6): 429-438. 10.7326/0003-4819-157-6-201209180-00537.CrossRefPubMed
12.
go back to reference Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S: Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ. 2013, 185 (4): E201-E211. 10.1503/cmaj.120744.CrossRefPubMedPubMedCentral Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S: Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ. 2013, 185 (4): E201-E211. 10.1503/cmaj.120744.CrossRefPubMedPubMedCentral
13.
go back to reference Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Rasmussen JV, Hilden J, Boutron I, Ravaud P, Brorson S: Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. Int J Epidemiol. 2014, 43 (3): 937-948. 10.1093/ije/dyt270.CrossRefPubMed Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Rasmussen JV, Hilden J, Boutron I, Ravaud P, Brorson S: Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. Int J Epidemiol. 2014, 43 (3): 937-948. 10.1093/ije/dyt270.CrossRefPubMed
14.
go back to reference Dechartres A, Boutron I, Roy C, Ravaud P: Inadequate planning and reporting of adjudication committees in clinical trials: recommendation proposal. J Clin Epidemiol. 2009, 62 (7): 695-702. 10.1016/j.jclinepi.2008.09.011.CrossRefPubMed Dechartres A, Boutron I, Roy C, Ravaud P: Inadequate planning and reporting of adjudication committees in clinical trials: recommendation proposal. J Clin Epidemiol. 2009, 62 (7): 695-702. 10.1016/j.jclinepi.2008.09.011.CrossRefPubMed
15.
go back to reference Tudur Smith C, Hickey H, Clarke M, Blazeby J, Williamson P: The trials methodological research agenda: results from a priority setting exercise. Trials. 2014, 15 (1): 32-10.1186/1745-6215-15-32.CrossRefPubMedPubMedCentral Tudur Smith C, Hickey H, Clarke M, Blazeby J, Williamson P: The trials methodological research agenda: results from a priority setting exercise. Trials. 2014, 15 (1): 32-10.1186/1745-6215-15-32.CrossRefPubMedPubMedCentral
16.
go back to reference Jairath V, Kahan BC, Gray A, Dore CJ, Mora A, Dyer C, Stokes EA, Llewelyn C, Bailey AA, Dallal H, Everett SM, James MW, Stanley AJ, Church N, Darwent M, Greenaway J, Le Jeune I, Reckless I, Campbell HE, Meredith S, Palmer KR, Logan RF, Travis SP, Walsh TS, Murphy MF: Restrictive vs liberal blood transfusion for acute upper gastrointestinal bleeding: rationale and protocol for a cluster randomized feasibility trial. Transfus Med Rev. 2013, 27 (3): 146-153. 10.1016/j.tmrv.2013.04.001.CrossRefPubMedPubMedCentral Jairath V, Kahan BC, Gray A, Dore CJ, Mora A, Dyer C, Stokes EA, Llewelyn C, Bailey AA, Dallal H, Everett SM, James MW, Stanley AJ, Church N, Darwent M, Greenaway J, Le Jeune I, Reckless I, Campbell HE, Meredith S, Palmer KR, Logan RF, Travis SP, Walsh TS, Murphy MF: Restrictive vs liberal blood transfusion for acute upper gastrointestinal bleeding: rationale and protocol for a cluster randomized feasibility trial. Transfus Med Rev. 2013, 27 (3): 146-153. 10.1016/j.tmrv.2013.04.001.CrossRefPubMedPubMedCentral
17.
go back to reference Kahan BC, Jairath V, Murphy MF, Dore CJ: Update on the transfusion in gastrointestinal bleeding (TRIGGER) trial: statistical analysis plan for a cluster-randomised feasibility trial. Trials. 2013, 14: 206-10.1186/1745-6215-14-206.CrossRefPubMedPubMedCentral Kahan BC, Jairath V, Murphy MF, Dore CJ: Update on the transfusion in gastrointestinal bleeding (TRIGGER) trial: statistical analysis plan for a cluster-randomised feasibility trial. Trials. 2013, 14: 206-10.1186/1745-6215-14-206.CrossRefPubMedPubMedCentral
18.
go back to reference Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K: A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009, 62 (5): 464-475. 10.1016/j.jclinepi.2008.12.011.CrossRefPubMed Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K: A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009, 62 (5): 464-475. 10.1016/j.jclinepi.2008.12.011.CrossRefPubMed
19.
go back to reference Putter H, Fiocco M, Geskus RB: Tutorial in biostatistics: competing risks and multi-state models. Stat Med. 2007, 26 (11): 2389-2430. 10.1002/sim.2712.CrossRefPubMed Putter H, Fiocco M, Geskus RB: Tutorial in biostatistics: competing risks and multi-state models. Stat Med. 2007, 26 (11): 2389-2430. 10.1002/sim.2712.CrossRefPubMed
20.
go back to reference Dresler CM, Olak J, Herndon JE, Richards WG, Scalzetti E, Fleishman SB, Kernstine KH, Demmy T, Jablons DM, Kohman L, Daniel TM, Haasler GB, Sugarbaker DJ: Phase III intergroup study of talc poudrage vs talc slurry sclerosis for malignant pleural effusion. Chest. 2005, 127 (3): 909-915. 10.1378/chest.127.3.909.CrossRefPubMedPubMedCentral Dresler CM, Olak J, Herndon JE, Richards WG, Scalzetti E, Fleishman SB, Kernstine KH, Demmy T, Jablons DM, Kohman L, Daniel TM, Haasler GB, Sugarbaker DJ: Phase III intergroup study of talc poudrage vs talc slurry sclerosis for malignant pleural effusion. Chest. 2005, 127 (3): 909-915. 10.1378/chest.127.3.909.CrossRefPubMedPubMedCentral
21.
go back to reference Villanueva C, Colomo A, Bosch A, Concepcion M, Hernandez-Gea V, Aracil C, Graupera I, Poca M, Alvarez-Urturi C, Gordillo J, Guarner-Argente C, Santalo M, Muniz E, Guarner C: Transfusion strategies for acute upper gastrointestinal bleeding. N Engl J Med. 2013, 368 (1): 11-21. 10.1056/NEJMoa1211801.CrossRefPubMed Villanueva C, Colomo A, Bosch A, Concepcion M, Hernandez-Gea V, Aracil C, Graupera I, Poca M, Alvarez-Urturi C, Gordillo J, Guarner-Argente C, Santalo M, Muniz E, Guarner C: Transfusion strategies for acute upper gastrointestinal bleeding. N Engl J Med. 2013, 368 (1): 11-21. 10.1056/NEJMoa1211801.CrossRefPubMed
22.
go back to reference Feagan BG, Sandborn WJ, D’Haens G, Pola S, McDonald JW, Rutgeerts P, Munkholm P, Mittmann U, King D, Wong CJ, Zou G, Donner A, Shackelton LM, Gilgen D, Nelson S, Vandervoort MK, Fahmy M, Loftus EV, Panaccione R, Travis SP, Van Assche GA, Vermeire S, Levesque BG: The role of centralized reading of endoscopy in a randomized controlled trial of mesalamine for ulcerative colitis. Gastroenterology. 2013, 145 (1): 149-157.e2. 10.1053/j.gastro.2013.03.025.CrossRefPubMed Feagan BG, Sandborn WJ, D’Haens G, Pola S, McDonald JW, Rutgeerts P, Munkholm P, Mittmann U, King D, Wong CJ, Zou G, Donner A, Shackelton LM, Gilgen D, Nelson S, Vandervoort MK, Fahmy M, Loftus EV, Panaccione R, Travis SP, Van Assche GA, Vermeire S, Levesque BG: The role of centralized reading of endoscopy in a randomized controlled trial of mesalamine for ulcerative colitis. Gastroenterology. 2013, 145 (1): 149-157.e2. 10.1053/j.gastro.2013.03.025.CrossRefPubMed
Metadata
Title
Reducing bias in open-label trials where blinded outcome assessment is not feasible: strategies from two randomised trials
Authors
Brennan C Kahan
Suzie Cro
Caroline J Doré
Daniel J Bratton
Sunita Rehal
Nick A Maskell
Najib Rahman
Vipul Jairath
Publication date
01-12-2014
Publisher
BioMed Central
Published in
Trials / Issue 1/2014
Electronic ISSN: 1745-6215
DOI
https://doi.org/10.1186/1745-6215-15-456

Other articles of this Issue 1/2014

Trials 1/2014 Go to the issue