Skip to main content
Top
Published in: Annals of Surgical Oncology 1/2021

01-01-2021 | Health Services Research and Global Oncology

Current Issues in Conduct and Reporting of Noninferiority Randomized Controlled Trials in Surgical Management of Cancer Patients

Authors: Armen Parsyan, MD, PhD, DrS, MPH, FRCSC, Wanda Marini, MD, MSc, PhD, Rouhi Fazelzad, BSc, MISt, David Moher, PhD, David McCready, MD, FRCSC, FACS

Published in: Annals of Surgical Oncology | Issue 1/2021

Login to get access

Abstract

Background

Serious concerns regarding quality of conduct and reporting of noninferiority trials (NITs) have been raised. Systematic analysis of the quality of the surgical NITs is lacking. Assessing the quality of conduct, reporting, and interpretation of surgical NITs in cancer patients is critical given their potential clinical impact. We aim to assess the quality of conduct, reporting, and interpretation of NITs that investigate the effects of surgical management in cancer patients.

Methods

A cross-sectional analysis of papers identified through a comprehensive literature database search was performed. Forty papers employing a phase III noninferiority (NI) randomized trial design to study effects of surgical methodology or sequencing of surgery in patients with solid cancers were included. Papers were assessed for type of analysis, justification of the noninferiority margin (NIM), consistency of type I error with confidence intervals (CIs), ability to achieve the predefined sample size, and interpretations regarding NI.

Results

Only half of the papers used both intention-to-treat and per protocol analyses; 62.5% provided no or poor justification for the NIM; 42.5% showed inconsistency of the type I error rate with CIs; 52.5% were deemed poor or fair quality, and 60.0% did not achieve the predefined sample size. One-fifth of the papers provided interpretation of the NI hypothesis that was not in concordance with the CONSORT guidelines.

Conclusions

The quality of conduct, reporting, and interpretation of surgical NITs is suboptimal, requiring further improvements through adherence to guidelines and rigorous assessment at the stages of the study approval, funding, and the peer-review process.
Appendix
Available only for authorised users
Literature
1.
go back to reference Huttner FJ, Capdeville L, Pianka F, et al. Systematic review of the quantity and quality of randomized clinical trials in pancreatic surgery. Br J Surg. Jan 2019;106(1):23–31.CrossRef Huttner FJ, Capdeville L, Pianka F, et al. Systematic review of the quantity and quality of randomized clinical trials in pancreatic surgery. Br J Surg. Jan 2019;106(1):23–31.CrossRef
2.
go back to reference Huttner FJ, Doerr-Harim C, Probst P, Tenckhoff S, Knebel P, Diener MK. Study methods in evidence-based surgery: methodological impediments and suggested approaches for the creation and transfer of knowledge in surgery. Eur Surg Res. 2014;53(1–4):86–94.CrossRef Huttner FJ, Doerr-Harim C, Probst P, Tenckhoff S, Knebel P, Diener MK. Study methods in evidence-based surgery: methodological impediments and suggested approaches for the creation and transfer of knowledge in surgery. Eur Surg Res. 2014;53(1–4):86–94.CrossRef
3.
go back to reference Huddart RA, Birtle A, Maynard L, et al. Clinical and patient-reported outcomes of SPARE—a randomised feasibility study of selective bladder preservation versus radical cystectomy. BJU Int. Nov 2017;120(5):639–50.CrossRef Huddart RA, Birtle A, Maynard L, et al. Clinical and patient-reported outcomes of SPARE—a randomised feasibility study of selective bladder preservation versus radical cystectomy. BJU Int. Nov 2017;120(5):639–50.CrossRef
4.
go back to reference Murthy VL, Desai NR, Vora A, Bhatt DL. Increasing proportion of clinical trials using noninferiority end points. Clin Cardiol. Sep 2012;35(9):522–3.CrossRef Murthy VL, Desai NR, Vora A, Bhatt DL. Increasing proportion of clinical trials using noninferiority end points. Clin Cardiol. Sep 2012;35(9):522–3.CrossRef
5.
go back to reference Suda KJ, Hurley AM, McKibbin T, Motl Moroney SE. Publication of noninferiority clinical trials: changes over a 20-year interval. Pharmacotherapy. 2011;31(9):833–9.CrossRef Suda KJ, Hurley AM, McKibbin T, Motl Moroney SE. Publication of noninferiority clinical trials: changes over a 20-year interval. Pharmacotherapy. 2011;31(9):833–9.CrossRef
6.
go back to reference Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG, Group C. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594–604.CrossRef Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG, Group C. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594–604.CrossRef
7.
go back to reference Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ, Group C. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295(10):1152–60.CrossRef Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ, Group C. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295(10):1152–60.CrossRef
8.
go back to reference FDA. Non-Inferiority Clinical Trials to Establish Effectiveness, Guidance for Industry. Food and Drug Administration, US Department of Health and Human Services. 2016. FDA. Non-Inferiority Clinical Trials to Establish Effectiveness, Guidance for Industry. Food and Drug Administration, US Department of Health and Human Services. 2016.
9.
go back to reference Aberegg SK, Hersh AM, Samore MH. Empirical consequences of current recommendations for the design and interpretation of noninferiority trials. J Gen Intern Med. 2018;33(1):88–96.CrossRef Aberegg SK, Hersh AM, Samore MH. Empirical consequences of current recommendations for the design and interpretation of noninferiority trials. J Gen Intern Med. 2018;33(1):88–96.CrossRef
10.
go back to reference Rehal S, Morris TP, Fielding K, Carpenter JR, Phillips PP. Non-inferiority trials: are they inferior? A systematic review of reporting in major medical journals. BMJ Open. Oct 7 2016;6(10):e012594.CrossRef Rehal S, Morris TP, Fielding K, Carpenter JR, Phillips PP. Non-inferiority trials: are they inferior? A systematic review of reporting in major medical journals. BMJ Open. Oct 7 2016;6(10):e012594.CrossRef
11.
go back to reference Turgeon RD, Reid EK, Rainkie DC. Design and interpretation of noninferiority trials. J Gen Intern Med. 2018;33(8):1215.CrossRef Turgeon RD, Reid EK, Rainkie DC. Design and interpretation of noninferiority trials. J Gen Intern Med. 2018;33(8):1215.CrossRef
12.
go back to reference Acuna SA, Dossa F, Baxter NN. Frequency of misinterpretation of inconclusive noninferiority trials: the case of the laparoscopic vs open resection for rectal cancer trials. JAMA Surg. 2019;154(1):90–2.CrossRef Acuna SA, Dossa F, Baxter NN. Frequency of misinterpretation of inconclusive noninferiority trials: the case of the laparoscopic vs open resection for rectal cancer trials. JAMA Surg. 2019;154(1):90–2.CrossRef
13.
14.
go back to reference Paesmans M, Grigoriu B, Ocak S, et al. Systematic qualitative review of randomised trials conducted in nonsmall cell lung cancer with a noninferiority or equivalence design. Eur Respir J. 2015;45(2):511–24.CrossRef Paesmans M, Grigoriu B, Ocak S, et al. Systematic qualitative review of randomised trials conducted in nonsmall cell lung cancer with a noninferiority or equivalence design. Eur Respir J. 2015;45(2):511–24.CrossRef
15.
go back to reference Fueglistaler P, Adamina M, Guller U. Non-inferiority trials in surgical oncology. Ann Surg Oncol. 2007;14(5):1532–9.CrossRef Fueglistaler P, Adamina M, Guller U. Non-inferiority trials in surgical oncology. Ann Surg Oncol. 2007;14(5):1532–9.CrossRef
16.
go back to reference 2017 Journal Impact Factor, Journal Citation Reports, ClarivateAnalytics, 2018. 2017 Journal Impact Factor, Journal Citation Reports, ClarivateAnalytics, 2018.
17.
go back to reference Acuna SA, Chesney TR, Amarasekera ST, Baxter NN. Defining non-inferiority margins for quality of surgical resection for rectal cancer: a Delphi consensus study. Ann Surg Oncol. 2018;25(11):3171–8.CrossRef Acuna SA, Chesney TR, Amarasekera ST, Baxter NN. Defining non-inferiority margins for quality of surgical resection for rectal cancer: a Delphi consensus study. Ann Surg Oncol. 2018;25(11):3171–8.CrossRef
18.
go back to reference Wyrwich KW, Spertus JA, Kroenke K, et al. Clinically important differences in health status for patients with heart disease: an expert consensus panel report. Am Heart J. 2004;147(4):615–22.CrossRef Wyrwich KW, Spertus JA, Kroenke K, et al. Clinically important differences in health status for patients with heart disease: an expert consensus panel report. Am Heart J. 2004;147(4):615–22.CrossRef
19.
go back to reference Bellamy N, Carette S, Ford PM, et al. Osteoarthritis antirheumatic drug trials. III. Setting the delta for clinical trials–results of a consensus development (Delphi) exercise. J Rheumatol. 1992;19(3):451–7.PubMed Bellamy N, Carette S, Ford PM, et al. Osteoarthritis antirheumatic drug trials. III. Setting the delta for clinical trials–results of a consensus development (Delphi) exercise. J Rheumatol. 1992;19(3):451–7.PubMed
20.
go back to reference McGlothlin AE, Lewis RJ. Minimal clinically important difference: defining what really matters to patients. JAMA. 2014;312(13):1342–3.CrossRef McGlothlin AE, Lewis RJ. Minimal clinically important difference: defining what really matters to patients. JAMA. 2014;312(13):1342–3.CrossRef
21.
go back to reference Acuna SA, Chesney TR, Baxter NN. Incorporating patient preferences in noninferiority trials. JAMA. 2019;322(4):305–6.CrossRef Acuna SA, Chesney TR, Baxter NN. Incorporating patient preferences in noninferiority trials. JAMA. 2019;322(4):305–6.CrossRef
22.
go back to reference Llewellyn-Thomas HA, Williams JI, Levy L, Naylor CD. Using a trade-off technique to assess patients’ treatment preferences for benign prostatic hyperplasia. Med Decis Making. 1996;16(3):262–82.CrossRef Llewellyn-Thomas HA, Williams JI, Levy L, Naylor CD. Using a trade-off technique to assess patients’ treatment preferences for benign prostatic hyperplasia. Med Decis Making. 1996;16(3):262–82.CrossRef
23.
go back to reference Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.CrossRef Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.CrossRef
24.
go back to reference D’Agostino RB, Sr., Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat Med. 2003;22(2):169–86.CrossRef D’Agostino RB, Sr., Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat Med. 2003;22(2):169–86.CrossRef
25.
go back to reference Shrier I, Verhagen E, Stovitz SD. The intention-to-treat analysis is not always the conservative approach. Am J Med. 2017;130(7):867–71.CrossRef Shrier I, Verhagen E, Stovitz SD. The intention-to-treat analysis is not always the conservative approach. Am J Med. 2017;130(7):867–71.CrossRef
26.
go back to reference Matsuyama Y. A comparison of the results of intent-to-treat, per-protocol, and g-estimation in the presence of non-random treatment changes in a time-to-event non-inferiority trial. Stat Med. 2010;29(20):2107–16.CrossRef Matsuyama Y. A comparison of the results of intent-to-treat, per-protocol, and g-estimation in the presence of non-random treatment changes in a time-to-event non-inferiority trial. Stat Med. 2010;29(20):2107–16.CrossRef
27.
go back to reference Head SJ, Kaul S, Bogers AJ, Kappetein AP. Non-inferiority study design: lessons to be learned from cardiovascular trials. Eur Heart J. 2012;33(11):1318–24.CrossRef Head SJ, Kaul S, Bogers AJ, Kappetein AP. Non-inferiority study design: lessons to be learned from cardiovascular trials. Eur Heart J. 2012;33(11):1318–24.CrossRef
28.
go back to reference Macaya F, Ryan N, Salinas P, Pocock SJ. Challenges in the design and interpretation of noninferiority trials: insights from recent stent trials. J Am Coll Cardiol. 2017;70(7):894–903.CrossRef Macaya F, Ryan N, Salinas P, Pocock SJ. Challenges in the design and interpretation of noninferiority trials: insights from recent stent trials. J Am Coll Cardiol. 2017;70(7):894–903.CrossRef
29.
go back to reference Schmid SL. Five years post-DORA: promoting best practices for research assessment. Mol Biol Cell. 2017;28(22):2941–4.CrossRef Schmid SL. Five years post-DORA: promoting best practices for research assessment. Mol Biol Cell. 2017;28(22):2941–4.CrossRef
30.
go back to reference Zhang L, Rousseau R, Sivertsen G. Science deserves to be judged by its contents, not by its wrapping: revisiting Seglen’s work on journal impact and research evaluation. PLoS One. 2017;12(3):e0174205.CrossRef Zhang L, Rousseau R, Sivertsen G. Science deserves to be judged by its contents, not by its wrapping: revisiting Seglen’s work on journal impact and research evaluation. PLoS One. 2017;12(3):e0174205.CrossRef
31.
go back to reference DORA. San Francisco Declaration on Research Assessment. December 16, 2012, Accessed June 15, 2019. DORA. San Francisco Declaration on Research Assessment. December 16, 2012, Accessed June 15, 2019.
32.
go back to reference Fung EK, Lore JM, Jr. Randomized controlled trials for evaluating surgical questions. Arch Otolaryngol Head Neck Surg. 2002;128(6):631–4.CrossRef Fung EK, Lore JM, Jr. Randomized controlled trials for evaluating surgical questions. Arch Otolaryngol Head Neck Surg. 2002;128(6):631–4.CrossRef
33.
go back to reference McLeod RS. Issues in surgical randomized controlled trials. World J Surg. 1999;23(12):1210–4.CrossRef McLeod RS. Issues in surgical randomized controlled trials. World J Surg. 1999;23(12):1210–4.CrossRef
34.
go back to reference Yu J, Chen W, Chen S, et al. Design, conduct, and analysis of surgical randomized controlled trials: a cross-sectional survey. Ann Surg. 2019;270(6):1065–9.CrossRef Yu J, Chen W, Chen S, et al. Design, conduct, and analysis of surgical randomized controlled trials: a cross-sectional survey. Ann Surg. 2019;270(6):1065–9.CrossRef
35.
go back to reference Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 Explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–37.CrossRef Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 Explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–37.CrossRef
36.
go back to reference Giuliano AE, Hunt KK, Ballman KV, et al. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA. 2011;305(6):569–75.CrossRef Giuliano AE, Hunt KK, Ballman KV, et al. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA. 2011;305(6):569–75.CrossRef
37.
go back to reference Donker M, van Tienhoven G, Straver ME, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981-22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol. 2014;15(12):1303–10.CrossRef Donker M, van Tienhoven G, Straver ME, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981-22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol. 2014;15(12):1303–10.CrossRef
38.
go back to reference Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci USA. 2018;115(11):2613–9.CrossRef Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci USA. 2018;115(11):2613–9.CrossRef
39.
go back to reference Boutron I, Haneef R, Yavchitz A, et al. Three randomized controlled trials evaluating the impact of “spin” in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit. BMC Med. 2019;17(1):105.CrossRef Boutron I, Haneef R, Yavchitz A, et al. Three randomized controlled trials evaluating the impact of “spin” in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit. BMC Med. 2019;17(1):105.CrossRef
40.
go back to reference Fleshman J, Branda M, Sargent DJ, et al. Effect of laparoscopic-assisted resection vs open resection of stage II or III rectal cancer on pathologic outcomes: the ACOSOG Z6051 randomized clinical trial. JAMA. 2015;314(13):1346–55.CrossRef Fleshman J, Branda M, Sargent DJ, et al. Effect of laparoscopic-assisted resection vs open resection of stage II or III rectal cancer on pathologic outcomes: the ACOSOG Z6051 randomized clinical trial. JAMA. 2015;314(13):1346–55.CrossRef
41.
go back to reference Stevenson AR, Solomon MJ, Lumley JW, et al. Effect of laparoscopic-assisted resection vs open resection on pathological outcomes in rectal cancer: the ALaCaRT randomized clinical trial. JAMA. 2015;314(13):1356–63.CrossRef Stevenson AR, Solomon MJ, Lumley JW, et al. Effect of laparoscopic-assisted resection vs open resection on pathological outcomes in rectal cancer: the ALaCaRT randomized clinical trial. JAMA. 2015;314(13):1356–63.CrossRef
Metadata
Title
Current Issues in Conduct and Reporting of Noninferiority Randomized Controlled Trials in Surgical Management of Cancer Patients
Authors
Armen Parsyan, MD, PhD, DrS, MPH, FRCSC
Wanda Marini, MD, MSc, PhD
Rouhi Fazelzad, BSc, MISt
David Moher, PhD
David McCready, MD, FRCSC, FACS
Publication date
01-01-2021
Publisher
Springer International Publishing
Published in
Annals of Surgical Oncology / Issue 1/2021
Print ISSN: 1068-9265
Electronic ISSN: 1534-4681
DOI
https://doi.org/10.1245/s10434-020-08575-7

Other articles of this Issue 1/2021

Annals of Surgical Oncology 1/2021 Go to the issue