Skip to main content
Top
Published in: Langenbeck's Archives of Surgery 3/2019

01-05-2019 | Abdominal Surgery | Systematic Reviews and Meta-analysis

Evidence-based recommendations for blinding in surgical trials

Authors: Pascal Probst, Steffen Zaschke, Patrick Heger, Julian C. Harnoss, Felix J. Hüttner, André L. Mihaljevic, Phillip Knebel, Markus K. Diener

Published in: Langenbeck's Archives of Surgery | Issue 3/2019

Login to get access

Abstract

Rationale

Blinding reduces performance and detection bias in randomized controlled trials (RCT). There is evidence that lack of blinding leads to overestimation of treatment effects in pharmacological trials. Since surgical trials use interventions with a physical component, blinding is often complicated. The aim of this study was to analyze, in general and abdominal surgery RCT, the status of blinding, the potential for blinding, and the influence of blinding on outcomes.

Methods

A systematic search of the literature in CENTRAL, MEDLINE, and Web of Science was conducted to identify RCT with a surgical intervention, starting in 1996, the year when the first CONSORT statement was published. Information on general study characteristics and blinding methods was extracted. The presence or absence of blinding of the study contributors—patients, surgeons, data collectors, outcome assessors, and data analysts—was analyzed. The association of blinding with the trial outcome was investigated for every study contributor.

Results

Out of 29,119 articles, 378 RCT were included in the analysis. These investigated a total of 62,522 patients, of whom 15,025 were blinded (24.0%). Contributors could have been blinded in far more trials, as the potential for blinding measures ranged from 69% for outcome assessors to 98% for data analysts. If blinding of surgeons would have been possible but had not been performed, this was associated with more significant trial outcomes (OR 13.670; 95% CI 1.308 to 142.840; p = 0.0289).

Discussion

The potential of blinding, an important quality measure in surgical RCT, has not been exhausted. This study summarizes the existing evidence on blinding in surgical trials and gives evidence-based recommendations for the use of blinding in future surgical trials.

Systematic review registration

PROSPERO 2015:CRD42015026837.
Appendix
Available only for authorised users
Literature
1.
3.
go back to reference Solomon MJ, McLeod RS (1995) Should we be performing more randomized controlled trials in evaluating surgical procedures? Surgery 118:459–467CrossRef Solomon MJ, McLeod RS (1995) Should we be performing more randomized controlled trials in evaluating surgical procedures? Surgery 118:459–467CrossRef
4.
go back to reference Cochrane Handbook, Higgins JPT (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from www.cochrane-handbook.org. Accessed 30.09.2016 Cochrane Handbook, Higgins JPT (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from www.​cochrane-handbook.​org. Accessed 30.09.2016
5.
go back to reference Schulz KF, Altman DG, Moher D, CONSORT Group (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Trials 11:32CrossRefPubMedPubMedCentral Schulz KF, Altman DG, Moher D, CONSORT Group (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Trials 11:32CrossRefPubMedPubMedCentral
6.
go back to reference Devereaux PJ, Manns BJ, Ghali WA, Quan H, Lacchetti C, Mouton VM et al (2001) Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA 285:2000–2003CrossRef Devereaux PJ, Manns BJ, Ghali WA, Quan H, Lacchetti C, Mouton VM et al (2001) Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA 285:2000–2003CrossRef
7.
8.
go back to reference Probst P, Grummich K, Harnoss JC, Hüttner FJ, Jensen K, Braun S, Kieser M, Ulrich A, Büchler MW, Diener MK (2016) Placebo-controlled trials in surgery: a systematic review and meta-analysis. Medicine (Baltimore) 95(17):e3516CrossRef Probst P, Grummich K, Harnoss JC, Hüttner FJ, Jensen K, Braun S, Kieser M, Ulrich A, Büchler MW, Diener MK (2016) Placebo-controlled trials in surgery: a systematic review and meta-analysis. Medicine (Baltimore) 95(17):e3516CrossRef
9.
go back to reference Boutron I, Tubach F, Giraudeau B, Ravaud P (2004) Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol 57(6):543–550CrossRefPubMed Boutron I, Tubach F, Giraudeau B, Ravaud P (2004) Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol 57(6):543–550CrossRefPubMed
10.
go back to reference Boutron I, Guittet L, Estellat C, Moher D, Hróbjartsson A, Ravaud P (2007) Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med 4(2):e61CrossRefPubMedPubMedCentral Boutron I, Guittet L, Estellat C, Moher D, Hróbjartsson A, Ravaud P (2007) Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med 4(2):e61CrossRefPubMedPubMedCentral
11.
go back to reference Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2010) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 8(5):336–341CrossRefPubMed Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2010) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 8(5):336–341CrossRefPubMed
12.
go back to reference Probst P, Grummich K, Heger P, Zaschke S, Knebel P, Ulrich A, Büchler MW, Diener MK (2016) Blinding in randomized controlled trials in general and abdominal surgery: protocol for a systematic review and empirical study. Syst Rev 5:48CrossRefPubMedPubMedCentral Probst P, Grummich K, Heger P, Zaschke S, Knebel P, Ulrich A, Büchler MW, Diener MK (2016) Blinding in randomized controlled trials in general and abdominal surgery: protocol for a systematic review and empirical study. Syst Rev 5:48CrossRefPubMedPubMedCentral
13.
go back to reference Goossen K, Tenckhoff S, Probst P, Grummich K, Mihaljevic AL, Büchler MW, Diener MK (2018) Optimal literature search for systematic reviews in surgery. Langenbeck’s Arch Surg 403(1):119–129CrossRef Goossen K, Tenckhoff S, Probst P, Grummich K, Mihaljevic AL, Büchler MW, Diener MK (2018) Optimal literature search for systematic reviews in surgery. Langenbeck’s Arch Surg 403(1):119–129CrossRef
14.
go back to reference Begg CB, Cho MK, Eastwood S et al (1996) Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 276:637–639CrossRef Begg CB, Cho MK, Eastwood S et al (1996) Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 276:637–639CrossRef
15.
go back to reference Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG (2004) Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291(20):2457–2465CrossRef Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG (2004) Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291(20):2457–2465CrossRef
16.
go back to reference Probst P, Grummich K, Tenckhoff S, Ulrich A, Büchler MW, Knebel P, Diener MK (2016) Industry bias in randomized controlled trials in general and abdominal surgery: an empirical study. Ann Surg 264(1):87–92CrossRefPubMed Probst P, Grummich K, Tenckhoff S, Ulrich A, Büchler MW, Knebel P, Diener MK (2016) Industry bias in randomized controlled trials in general and abdominal surgery: an empirical study. Ann Surg 264(1):87–92CrossRefPubMed
17.
go back to reference Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP (2006) Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol 59(7):697–703CrossRefPubMed Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP (2006) Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol 59(7):697–703CrossRefPubMed
19.
go back to reference Park JS, Kim JH, Kim JK, Yoon DS (2015) The role of abdominal drainage to prevent of intra-abdominal complications after laparoscopic cholecystectomy for acute cholecystitis: prospective randomized trial. Surg Endosc 29(2):453–457CrossRefPubMed Park JS, Kim JH, Kim JK, Yoon DS (2015) The role of abdominal drainage to prevent of intra-abdominal complications after laparoscopic cholecystectomy for acute cholecystitis: prospective randomized trial. Surg Endosc 29(2):453–457CrossRefPubMed
20.
go back to reference Diener MK, Knebel P, Kieser M, Schüler P, Schiergens TS, Atanassov V, Neudecker J, Stein E, Thielemann H, Kunz R, von Frankenberg M, Schernikau U, Bunse J, Jansen-Winkeln B, Partecke LI, Prechtl G, Pochhammer J, Bouchard R, Hodina R, Beckurts KT, Leißner L, Lemmens HP, Kallinowski F, Thomusch O, Seehofer D, Simon T, Hyhlik-Dürr A, Seiler CM, Hackert T, Reissfelder C, Hennig R, Doerr-Harim C, Klose C, Ulrich A, Büchler MW (2014) Effectiveness of triclosan-coated PDS Plus versus uncoated PDS II sutures for prevention of surgical site infection after abdominal wall closure: the randomised controlled PROUD trial. Lanct 384(9938):142–152CrossRef Diener MK, Knebel P, Kieser M, Schüler P, Schiergens TS, Atanassov V, Neudecker J, Stein E, Thielemann H, Kunz R, von Frankenberg M, Schernikau U, Bunse J, Jansen-Winkeln B, Partecke LI, Prechtl G, Pochhammer J, Bouchard R, Hodina R, Beckurts KT, Leißner L, Lemmens HP, Kallinowski F, Thomusch O, Seehofer D, Simon T, Hyhlik-Dürr A, Seiler CM, Hackert T, Reissfelder C, Hennig R, Doerr-Harim C, Klose C, Ulrich A, Büchler MW (2014) Effectiveness of triclosan-coated PDS Plus versus uncoated PDS II sutures for prevention of surgical site infection after abdominal wall closure: the randomised controlled PROUD trial. Lanct 384(9938):142–152CrossRef
21.
go back to reference Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273:408–412CrossRefPubMed Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273:408–412CrossRefPubMed
22.
go back to reference Montori VM, Bhandari M, Devereaux PJ, Manns BJ, Ghali WA, Guyatt GH (2002) In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol 55:787–790CrossRefPubMed Montori VM, Bhandari M, Devereaux PJ, Manns BJ, Ghali WA, Guyatt GH (2002) In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol 55:787–790CrossRefPubMed
26.
27.
go back to reference Karanicolas PJ, Farrokhyar F, Bhandari M (2010) Practical tips for surgical research: blinding: who, what, when, why, how? Can J Surg 53:345–348PubMedPubMedCentral Karanicolas PJ, Farrokhyar F, Bhandari M (2010) Practical tips for surgical research: blinding: who, what, when, why, how? Can J Surg 53:345–348PubMedPubMedCentral
28.
go back to reference Farrokhyar F, Bajammal S, Kahnamoui K, Bhandari M (2010) Practical tips for surgical research. Ensuring balanced groups in surgical trials. Can J Surg 53:418–423PubMedPubMedCentral Farrokhyar F, Bajammal S, Kahnamoui K, Bhandari M (2010) Practical tips for surgical research. Ensuring balanced groups in surgical trials. Can J Surg 53:418–423PubMedPubMedCentral
31.
34.
go back to reference Berkman ND, Santaguida PL, Viswanathan M, Morton SC (2014) In: The empirical evidence of Bias in trials measuring treatment differences, Rockville (MD): Agency for healthcare research and quality (US), Report No.: 14-EHC050-EF. AHRQ Methods for effective health care. Berkman ND, Santaguida PL, Viswanathan M, Morton SC (2014) In: The empirical evidence of Bias in trials measuring treatment differences, Rockville (MD): Agency for healthcare research and quality (US), Report No.: 14-EHC050-EF. AHRQ Methods for effective health care.
38.
go back to reference Metcalfe C, Avery K, Berrisford R, Barham P, Noble SM, Fernandez AM, Hanna G, Goldin R, Elliott J, Wheatley T, Sanders G, Hollowood A, Falk S, Titcomb D, Streets C, Donovan JL, Blazeby JM (2016) Comparing open and minimally invasive surgical procedures for oesophagectomy in the treatment of cancer: the ROMIO (Randomised Oesophagectomy: Minimally Invasive or Open) feasibility study and pilot trial. Health Technol Assess 20:1–68. https://doi.org/10.3310/hta20480 CrossRefPubMedPubMedCentral Metcalfe C, Avery K, Berrisford R, Barham P, Noble SM, Fernandez AM, Hanna G, Goldin R, Elliott J, Wheatley T, Sanders G, Hollowood A, Falk S, Titcomb D, Streets C, Donovan JL, Blazeby JM (2016) Comparing open and minimally invasive surgical procedures for oesophagectomy in the treatment of cancer: the ROMIO (Randomised Oesophagectomy: Minimally Invasive or Open) feasibility study and pilot trial. Health Technol Assess 20:1–68. https://​doi.​org/​10.​3310/​hta20480 CrossRefPubMedPubMedCentral
40.
go back to reference Sim J (2017) Outcome-adaptive randomization: some ethical issues. Trials 18(Suppl 1):O42 Sim J (2017) Outcome-adaptive randomization: some ethical issues. Trials 18(Suppl 1):O42
41.
go back to reference Elbers R, Savovic J, Blencowe N, Higgins JPT, Sterne JAC (2017) Performance bias in trials that cannot blind participants and healthcare providers to assigned interventions: implications for trial conduct. Trials 18(Suppl 1):P478 Elbers R, Savovic J, Blencowe N, Higgins JPT, Sterne JAC (2017) Performance bias in trials that cannot blind participants and healthcare providers to assigned interventions: implications for trial conduct. Trials 18(Suppl 1):P478
42.
go back to reference Cousins S, Chalmers K, Avery K, Blencowe N, Brookes S, Blazeby J, Kobetic M, Munder T (2017) Minimising bias in surgical RCTs through blinding: a systematic review. Trials 18(Suppl 1):P377 Cousins S, Chalmers K, Avery K, Blencowe N, Brookes S, Blazeby J, Kobetic M, Munder T (2017) Minimising bias in surgical RCTs through blinding: a systematic review. Trials 18(Suppl 1):P377
43.
go back to reference Blencowe N, Main BG, Blazeby JM (2017) Beyond blinding: a systematic review to explore performance bias in surgical RCTs. Trials 18(Suppl 1):P224 Blencowe N, Main BG, Blazeby JM (2017) Beyond blinding: a systematic review to explore performance bias in surgical RCTs. Trials 18(Suppl 1):P224
44.
go back to reference Bassi C, Marchegiani G, Dervenis C, Sarr M, Abu Hilal M, Adham M, Allen P, Andersson R, Asbun HJ, Besselink MG, Conlon K, Del Chiaro M, Falconi M, Fernandez-Cruz L, Fernandez-Del Castillo C, Fingerhut A, Friess H, Gouma DJ, Hackert T, Izbicki J, Lillemoe KD, Neoptolemos JP, Olah A, Schulick R, Shrikhande SV, Takada T, Takaori K, Traverso W, Vollmer CR, Wolfgang CL, Yeo CJ, Salvia R, Buchler M, International Study Group on Pancreatic Surgery (ISGPS) (2017) The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 years after. Surgery 161(3):584–591CrossRef Bassi C, Marchegiani G, Dervenis C, Sarr M, Abu Hilal M, Adham M, Allen P, Andersson R, Asbun HJ, Besselink MG, Conlon K, Del Chiaro M, Falconi M, Fernandez-Cruz L, Fernandez-Del Castillo C, Fingerhut A, Friess H, Gouma DJ, Hackert T, Izbicki J, Lillemoe KD, Neoptolemos JP, Olah A, Schulick R, Shrikhande SV, Takada T, Takaori K, Traverso W, Vollmer CR, Wolfgang CL, Yeo CJ, Salvia R, Buchler M, International Study Group on Pancreatic Surgery (ISGPS) (2017) The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 years after. Surgery 161(3):584–591CrossRef
Metadata
Title
Evidence-based recommendations for blinding in surgical trials
Authors
Pascal Probst
Steffen Zaschke
Patrick Heger
Julian C. Harnoss
Felix J. Hüttner
André L. Mihaljevic
Phillip Knebel
Markus K. Diener
Publication date
01-05-2019
Publisher
Springer Berlin Heidelberg
Published in
Langenbeck's Archives of Surgery / Issue 3/2019
Print ISSN: 1435-2443
Electronic ISSN: 1435-2451
DOI
https://doi.org/10.1007/s00423-019-01761-6

Other articles of this Issue 3/2019

Langenbeck's Archives of Surgery 3/2019 Go to the issue