Skip to main content
Top
Published in: Clinical Orthopaedics and Related Research® 9/2011

01-09-2011 | Basic Research

Rationale for and Methods of Superiority, Noninferiority, or Equivalence Designs in Orthopaedic, Controlled Trials

Author: Patrick Vavken, MD, MSc, FRSPH

Published in: Clinical Orthopaedics and Related Research® | Issue 9/2011

Login to get access

Abstract

Background

To provide value-based healthcare in orthopaedics, controlled trials are needed to assess the comparative effectiveness of treatments. Typically comparative trials are based on superiority testing using statistical tests that produce a p value. However, as orthopaedic treatments continue to improve, superiority becomes more difficult to show and, perhaps, less important as margins of improvement shrink to clinically irrelevant levels. Alternative methods to compare groups in controlled trials are noninferiority and equivalence. It is important to equip the reader of the orthopaedic literature with the knowledge to understand and critically evaluate the methods and findings of trials attempting to establish superiority, noninferiority, and equivalence.

Questions/purposes

I will discuss supplemental and alternative methods to superiority for assessment of the outcome of controlled trials in the context of diminishing returns on new therapies over old ones.

Methods

The three methods—superiority, noninferiority, and equivalence—are presented and compared, with a discussion of implied pitfalls and problems.

Results

Noninferiority and equivalence offer alternatives to superiority testing and allow one to judge whether a new treatment is no worse (within a margin) or substantively the same as an active control. Noninferiority testing also allows for inclusion of superiority testing in the same study without the need for adjustment of the statistical methods.

Conclusions

Noninferiority and equivalence testing might prove most valuable in orthopaedic, controlled trials as they allow for comparative assessment of treatments with similar primary end points but potentially important differences in secondary outcomes, safety profiles, and cost-effectiveness.
Literature
1.
go back to reference Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995;311:485.PubMed Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995;311:485.PubMed
2.
go back to reference Avisory Board. Reducing orthopedic implant costs. Washington, DC: The Advisory Board; 2004. Avisory Board. Reducing orthopedic implant costs. Washington, DC: The Advisory Board; 2004.
3.
go back to reference Bauwens K, Matthes G, Wich M, Gebhard F, Hanson B, Ekkernkamp A, Stengel D. Navigated total knee replacement: a meta-analysis. J Bone Joint Surg Am. 2007;89:261–269.PubMedCrossRef Bauwens K, Matthes G, Wich M, Gebhard F, Hanson B, Ekkernkamp A, Stengel D. Navigated total knee replacement: a meta-analysis. J Bone Joint Surg Am. 2007;89:261–269.PubMedCrossRef
4.
go back to reference Bhandari M, Giannoudis P. Evidence-based medicine: what it is and what it is not. Injury. 2006;37:302–306.PubMedCrossRef Bhandari M, Giannoudis P. Evidence-based medicine: what it is and what it is not. Injury. 2006;37:302–306.PubMedCrossRef
5.
go back to reference Bhandari M, Richards RR, Sprague S, Schemitsch EH. The quality of reporting of randomized trials in the Journal of Bone and Joint Surgery from 1988 through 2000. J Bone Joint Surg Am. 2002;84:388–396.PubMedCrossRef Bhandari M, Richards RR, Sprague S, Schemitsch EH. The quality of reporting of randomized trials in the Journal of Bone and Joint Surgery from 1988 through 2000. J Bone Joint Surg Am. 2002;84:388–396.PubMedCrossRef
6.
go back to reference Bubbar VK, Kreder HJ. The intention-to-treat principle: a primer for the orthopaedic surgeon. J Bone Joint Surg Am. 2006;88:2097–2099.PubMedCrossRef Bubbar VK, Kreder HJ. The intention-to-treat principle: a primer for the orthopaedic surgeon. J Bone Joint Surg Am. 2006;88:2097–2099.PubMedCrossRef
7.
go back to reference Burns LR, Housman MG, Booth RE Jr, Koenig A. Implant vendors and hospitals: competing influences over product choice by orthopedic surgeons. Health Care Manage Rev. 2009;34:2–18.PubMed Burns LR, Housman MG, Booth RE Jr, Koenig A. Implant vendors and hospitals: competing influences over product choice by orthopedic surgeons. Health Care Manage Rev. 2009;34:2–18.PubMed
8.
go back to reference Busija L, Osborne RH, Nilsdotter A, Buchbinder R, Roos EM. Magnitude and meaningfulness of change in SF-36 scores in four types of orthopedic surgery. Health Qual Life Outcomes. 2008;6:55.PubMedCrossRef Busija L, Osborne RH, Nilsdotter A, Buchbinder R, Roos EM. Magnitude and meaningfulness of change in SF-36 scores in four types of orthopedic surgery. Health Qual Life Outcomes. 2008;6:55.PubMedCrossRef
10.
go back to reference D’Agostino RB Sr, Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues: the encounters of academic consultants in statistics. Stat Med. 2003;22:169–186.PubMedCrossRef D’Agostino RB Sr, Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues: the encounters of academic consultants in statistics. Stat Med. 2003;22:169–186.PubMedCrossRef
11.
go back to reference Dimick JB, Diener-West M, Lipsett PA. Negative results of randomized clinical trials published in the surgical literature: equivalency or error? Arch Surg. 2001;136:796–800.PubMedCrossRef Dimick JB, Diener-West M, Lipsett PA. Negative results of randomized clinical trials published in the surgical literature: equivalency or error? Arch Surg. 2001;136:796–800.PubMedCrossRef
12.
go back to reference Dorey FJ. In brief: The p value: what is it and what does it tell you? Clin Orthop Relat Res. 2010;468:2297–2298.PubMedCrossRef Dorey FJ. In brief: The p value: what is it and what does it tell you? Clin Orthop Relat Res. 2010;468:2297–2298.PubMedCrossRef
13.
go back to reference Dorey FJ. In brief: statistics in brief: confidence intervals: what is the real result in the target population? Clin Orthop Relat Res. 2010;468:3137–3138.PubMedCrossRef Dorey FJ. In brief: statistics in brief: confidence intervals: what is the real result in the target population? Clin Orthop Relat Res. 2010;468:3137–3138.PubMedCrossRef
14.
go back to reference Eskelinen A, Paavolainen P, Helenius I, Pulkkinen P, Remes V. Total hip arthroplasty for rheumatoid arthritis in younger patients: 2,557 replacements in the Finnish Arthroplasty Register followed for 0–24 years. Acta Orthop. 2006;77:853–865.PubMedCrossRef Eskelinen A, Paavolainen P, Helenius I, Pulkkinen P, Remes V. Total hip arthroplasty for rheumatoid arthritis in younger patients: 2,557 replacements in the Finnish Arthroplasty Register followed for 0–24 years. Acta Orthop. 2006;77:853–865.PubMedCrossRef
15.
go back to reference Eskelinen A, Remes V, Helenius I, Pulkkinen P, Nevalainen J, Paavolainen P. Total hip arthroplasty for primary osteoarthrosis in younger patients in the Finnish arthroplasty register: 4,661 primary replacements followed for 0–22 years. Acta Orthop. 2005;76:28–41.PubMedCrossRef Eskelinen A, Remes V, Helenius I, Pulkkinen P, Nevalainen J, Paavolainen P. Total hip arthroplasty for primary osteoarthrosis in younger patients in the Finnish arthroplasty register: 4,661 primary replacements followed for 0–22 years. Acta Orthop. 2005;76:28–41.PubMedCrossRef
16.
go back to reference Everson-Stewart S, Emerson SS. Bio-creep in non-inferiority clinical trials. Stat Med. 2010;29:2769–2780.PubMedCrossRef Everson-Stewart S, Emerson SS. Bio-creep in non-inferiority clinical trials. Stat Med. 2010;29:2769–2780.PubMedCrossRef
17.
go back to reference Fisher RA. Statistical Methods for Research Workers. Edinburgh, UK: Oliver and Boyd; 1925. Fisher RA. Statistical Methods for Research Workers. Edinburgh, UK: Oliver and Boyd; 1925.
19.
go back to reference Freedman KB, Back S, Bernstein J. Sample size and statistical power of randomised. controlled trials in orthopaedics. J Bone Joint Surg Br. 2001;83:397–402.PubMedCrossRef Freedman KB, Back S, Bernstein J. Sample size and statistical power of randomised. controlled trials in orthopaedics. J Bone Joint Surg Br. 2001;83:397–402.PubMedCrossRef
20.
21.
go back to reference Greene WL, Concato J, Feinstein AR. Claims of equivalence in medical research: are they supported by the evidence? Ann Intern Med. 2000;132:715–722.PubMed Greene WL, Concato J, Feinstein AR. Claims of equivalence in medical research: are they supported by the evidence? Ann Intern Med. 2000;132:715–722.PubMed
22.
go back to reference Hart R, Janecek M, Cizmar I, Stipcak V, Kucera B, Filan P. [Minimally invasive and navigated implantation for total knee arthroplasty: X-ray analysis and early clinical results] [in German]. Orthopade. 2006;35:552–557.PubMedCrossRef Hart R, Janecek M, Cizmar I, Stipcak V, Kucera B, Filan P. [Minimally invasive and navigated implantation for total knee arthroplasty: X-ray analysis and early clinical results] [in German]. Orthopade. 2006;35:552–557.PubMedCrossRef
23.
go back to reference Hochberg MC. Structure-modifying effects of chondroitin sulfate in knee osteoarthritis: an updated meta-analysis of randomized placebo-controlled trials of 2-year duration. Osteoarthritis Cartilage. 2010;18(suppl 1):S28–S31.PubMedCrossRef Hochberg MC. Structure-modifying effects of chondroitin sulfate in knee osteoarthritis: an updated meta-analysis of randomized placebo-controlled trials of 2-year duration. Osteoarthritis Cartilage. 2010;18(suppl 1):S28–S31.PubMedCrossRef
24.
go back to reference Hung HM, Wang SJ, O’Neill R. A regulatory perspective on choice of margin and statistical inference issue in non-inferiority trials. Biom J. 2005;47:28–36; discussion 99–107.PubMedCrossRef Hung HM, Wang SJ, O’Neill R. A regulatory perspective on choice of margin and statistical inference issue in non-inferiority trials. Biom J. 2005;47:28–36; discussion 99–107.PubMedCrossRef
25.
go back to reference Hung HM, Wang SJ, O’Neill R. Challenges and regulatory experiences with non-inferiority trial design without placebo arm. Biom J. 2009;51:324–334.PubMedCrossRef Hung HM, Wang SJ, O’Neill R. Challenges and regulatory experiences with non-inferiority trial design without placebo arm. Biom J. 2009;51:324–334.PubMedCrossRef
26.
go back to reference Kocher MS, Zurakowski D. Clinical epidemiology and biostatistics: a primer for orthopaedic surgeons. J Bone Joint Surg Am. 2004;86:607–620.PubMed Kocher MS, Zurakowski D. Clinical epidemiology and biostatistics: a primer for orthopaedic surgeons. J Bone Joint Surg Am. 2004;86:607–620.PubMed
27.
28.
go back to reference Lesaffre E. Superiority, equivalence, and non-inferiority trials. Bull NYU Hosp Jt Dis. 2008;66:150–154.PubMed Lesaffre E. Superiority, equivalence, and non-inferiority trials. Bull NYU Hosp Jt Dis. 2008;66:150–154.PubMed
29.
go back to reference Lesaffre E. Use and misuse of the p-value. Bull NYU Hosp Jt Dis. 2008;66:146–149.PubMed Lesaffre E. Use and misuse of the p-value. Bull NYU Hosp Jt Dis. 2008;66:146–149.PubMed
30.
go back to reference Lochner HV, Bhandari M, Tornetta P 3rd. Type-II error rates (beta errors) of randomized trials in orthopaedic trauma. J Bone Joint Surg Am. 2001;83:1650–1655.PubMed Lochner HV, Bhandari M, Tornetta P 3rd. Type-II error rates (beta errors) of randomized trials in orthopaedic trauma. J Bone Joint Surg Am. 2001;83:1650–1655.PubMed
31.
go back to reference Makela KT, Eskelinen A, Paavolainen P, Pulkkinen P, Remes V. Cementless total hip arthroplasty for primary osteoarthritis in patients aged 55 years and older. Acta Orthop. 2010;81:42–52.PubMedCrossRef Makela KT, Eskelinen A, Paavolainen P, Pulkkinen P, Remes V. Cementless total hip arthroplasty for primary osteoarthritis in patients aged 55 years and older. Acta Orthop. 2010;81:42–52.PubMedCrossRef
32.
go back to reference Makela KT, Eskelinen A, Pulkkinen P, Paavolainen P, Remes V. Total hip arthroplasty for primary osteoarthritis in patients fifty-five years of age or older: an analysis of the Finnish arthroplasty registry. J Bone Joint Surg Am. 2008;90:2160–2170.PubMedCrossRef Makela KT, Eskelinen A, Pulkkinen P, Paavolainen P, Remes V. Total hip arthroplasty for primary osteoarthritis in patients fifty-five years of age or older: an analysis of the Finnish arthroplasty registry. J Bone Joint Surg Am. 2008;90:2160–2170.PubMedCrossRef
33.
go back to reference March LM, Cross MJ, Lapsley H, Brnabic AJ, Tribe KL, Bachmeier CJ, Courtenay BG, Brooks PM. Outcomes after hip or knee replacement surgery for osteoarthritis: a prospective cohort study comparing patients’ quality of life before and after surgery with age-related population norms. Med J Aust. 1999;171:235–238.PubMed March LM, Cross MJ, Lapsley H, Brnabic AJ, Tribe KL, Bachmeier CJ, Courtenay BG, Brooks PM. Outcomes after hip or knee replacement surgery for osteoarthritis: a prospective cohort study comparing patients’ quality of life before and after surgery with age-related population norms. Med J Aust. 1999;171:235–238.PubMed
34.
go back to reference Matilde Sanchez M, Chen X. Choosing the analysis population in non-inferiority studies: per protocol or intent-to-treat. Stat Med. 2006;25:1169–1181.PubMedCrossRef Matilde Sanchez M, Chen X. Choosing the analysis population in non-inferiority studies: per protocol or intent-to-treat. Stat Med. 2006;25:1169–1181.PubMedCrossRef
35.
go back to reference Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357:1191–1194.PubMedCrossRef Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357:1191–1194.PubMedCrossRef
36.
go back to reference Morikawa T, Yoshida M. A useful testing strategy in phase III trials: combined test of superiority and test of equivalence. J Biopharm Stat. 1995;5:297–306.PubMedCrossRef Morikawa T, Yoshida M. A useful testing strategy in phase III trials: combined test of superiority and test of equivalence. J Biopharm Stat. 1995;5:297–306.PubMedCrossRef
37.
go back to reference Moye L. Multiple Analyses in Clinical Trials: Fundamentals for Investigators. New York, NY: Springer; 2003. Moye L. Multiple Analyses in Clinical Trials: Fundamentals for Investigators. New York, NY: Springer; 2003.
38.
go back to reference Ohrn F, Jennison C. Optimal group-sequential designs for simultaneous testing of superiority and non-inferiority. Stat Med. 2010;29:743–759.PubMedCrossRef Ohrn F, Jennison C. Optimal group-sequential designs for simultaneous testing of superiority and non-inferiority. Stat Med. 2010;29:743–759.PubMedCrossRef
39.
go back to reference Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ; CONSORT Group. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295:1152–1160.PubMedCrossRef Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ; CONSORT Group. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295:1152–1160.PubMedCrossRef
40.
41.
go back to reference Poolman RW, Struijs PA, Krips R, Sierevelt IN, Lutz KH, Bhandari M. Does a “Level I Evidence” rating imply high quality of reporting in orthopaedic randomised controlled trials? BMC Med Res Methodol. 2006;6:44.PubMedCrossRef Poolman RW, Struijs PA, Krips R, Sierevelt IN, Lutz KH, Bhandari M. Does a “Level I Evidence” rating imply high quality of reporting in orthopaedic randomised controlled trials? BMC Med Res Methodol. 2006;6:44.PubMedCrossRef
42.
43.
44.
go back to reference Rothmann M, Li N, Chen G, Chi GY, Temple R, Tsou HH. Design and analysis of non-inferiority mortality trials in oncology. Stat Med. 2003;22:239–264.PubMedCrossRef Rothmann M, Li N, Chen G, Chi GY, Temple R, Tsou HH. Design and analysis of non-inferiority mortality trials in oncology. Stat Med. 2003;22:239–264.PubMedCrossRef
45.
go back to reference Scott IA. Non-inferiority trials: determining whether alternative treatments are good enough. Med J Aust. 2009;190:326–330.PubMed Scott IA. Non-inferiority trials: determining whether alternative treatments are good enough. Med J Aust. 2009;190:326–330.PubMed
46.
go back to reference Simoens S. Health economics of medical devices: opportunities and challenges. J Med Econ. 2008;11:713–717.PubMedCrossRef Simoens S. Health economics of medical devices: opportunities and challenges. J Med Econ. 2008;11:713–717.PubMedCrossRef
47.
go back to reference Steinijans VW, Neuhauser M, Bretz F. Equivalence concepts in clinical trials. Eur J Drug Metab Pharmacokinet. 2000;25:38–40.PubMedCrossRef Steinijans VW, Neuhauser M, Bretz F. Equivalence concepts in clinical trials. Eur J Drug Metab Pharmacokinet. 2000;25:38–40.PubMedCrossRef
48.
go back to reference Sterne JA, Davey Smith G. Sifting the evidence: what’s wrong with significance texts. BMJ. 2001;322:226–231.PubMedCrossRef Sterne JA, Davey Smith G. Sifting the evidence: what’s wrong with significance texts. BMJ. 2001;322:226–231.PubMedCrossRef
49.
go back to reference Streiner DL. Alternatives to placebo-controlled trials. Can J Neurol Sci. 2007;34(Suppl 1):S37–S41.PubMed Streiner DL. Alternatives to placebo-controlled trials. Can J Neurol Sci. 2007;34(Suppl 1):S37–S41.PubMed
50.
go back to reference Student. The probable error of the mean. Biometrika. 1908;6:1–25. Student. The probable error of the mean. Biometrika. 1908;6:1–25.
51.
go back to reference Student. On testing varieties of cereal. Biometrika. 1923;15:271–293. Student. On testing varieties of cereal. Biometrika. 1923;15:271–293.
52.
go back to reference Tashiro Y, Miura H, Matsuda S, Okazaki K, Iwamoto Y. Minimally invasive versus standard approach in total knee arthroplasty. Clin Orthop Relat Res. 2007;463:144–150.PubMed Tashiro Y, Miura H, Matsuda S, Okazaki K, Iwamoto Y. Minimally invasive versus standard approach in total knee arthroplasty. Clin Orthop Relat Res. 2007;463:144–150.PubMed
53.
go back to reference Tsong Y, Wang SJ, Hung HM, Cui L. Statistical issues on objective, design, and analysis of noninferiority active-controlled clinical trial. J Biopharm Stat. 2003;13:29–41.PubMedCrossRef Tsong Y, Wang SJ, Hung HM, Cui L. Statistical issues on objective, design, and analysis of noninferiority active-controlled clinical trial. J Biopharm Stat. 2003;13:29–41.PubMedCrossRef
54.
go back to reference Tuma RS. Trend toward noninferiority trials may mean more difficult interpretation of trial results. J Natl Cancer Inst. 2007;99:1746–1748.PubMedCrossRef Tuma RS. Trend toward noninferiority trials may mean more difficult interpretation of trial results. J Natl Cancer Inst. 2007;99:1746–1748.PubMedCrossRef
55.
go back to reference Vavken P, Castellani L, Sculco TP. Prophylaxis of heterotopic ossification of the hip: systematic review and meta-analysis. Clin Orthop Relat Res. 2009;467:3283–3289.PubMedCrossRef Vavken P, Castellani L, Sculco TP. Prophylaxis of heterotopic ossification of the hip: systematic review and meta-analysis. Clin Orthop Relat Res. 2009;467:3283–3289.PubMedCrossRef
56.
go back to reference Vavken P, Culen G, Dorotka R. [Clinical applicability of evidence-based orthopedics: a cross-sectional study of the quality of orthopedic evidence] [in German]. Z Orthop Unfall. 2008;146:21–25.PubMedCrossRef Vavken P, Culen G, Dorotka R. [Clinical applicability of evidence-based orthopedics: a cross-sectional study of the quality of orthopedic evidence] [in German]. Z Orthop Unfall. 2008;146:21–25.PubMedCrossRef
57.
go back to reference Vavken P, Culen G, Dorotka R. Management of confounding in controlled orthopaedic trials: a cross-sectional study. Clin Orthop Relat Res. 2008;466:985–989.PubMedCrossRef Vavken P, Culen G, Dorotka R. Management of confounding in controlled orthopaedic trials: a cross-sectional study. Clin Orthop Relat Res. 2008;466:985–989.PubMedCrossRef
58.
go back to reference Vavken P, Dorotka R. A systematic review of conflicting meta-analyses in orthopaedic surgery. Clin Orthop Relat Res. 2009;467:2723–2735.PubMedCrossRef Vavken P, Dorotka R. A systematic review of conflicting meta-analyses in orthopaedic surgery. Clin Orthop Relat Res. 2009;467:2723–2735.PubMedCrossRef
59.
go back to reference Vavken P, Heinrich KM, Koppelhuber C, Rois S, Dorotka R. The use of confidence intervals in reporting orthopaedic research findings. Clin Orthop Relat Res. 2009;467:3334–3339.PubMedCrossRef Vavken P, Heinrich KM, Koppelhuber C, Rois S, Dorotka R. The use of confidence intervals in reporting orthopaedic research findings. Clin Orthop Relat Res. 2009;467:3334–3339.PubMedCrossRef
60.
go back to reference Vavken P, Kotz R, Dorotka R. [Minimally invasive hip replacement: a meta-analysis] [in German]. Z Orthop Unfall. 2007;145:152–156.PubMedCrossRef Vavken P, Kotz R, Dorotka R. [Minimally invasive hip replacement: a meta-analysis] [in German]. Z Orthop Unfall. 2007;145:152–156.PubMedCrossRef
61.
go back to reference Wang SJ, Hung HM, Tsong Y, Cui L. Group sequential test strategies for superiority and non-inferiority hypotheses in active controlled clinical trials. Stat Med. 2001;20:1903–1912.PubMedCrossRef Wang SJ, Hung HM, Tsong Y, Cui L. Group sequential test strategies for superiority and non-inferiority hypotheses in active controlled clinical trials. Stat Med. 2001;20:1903–1912.PubMedCrossRef
Metadata
Title
Rationale for and Methods of Superiority, Noninferiority, or Equivalence Designs in Orthopaedic, Controlled Trials
Author
Patrick Vavken, MD, MSc, FRSPH
Publication date
01-09-2011
Publisher
Springer-Verlag
Published in
Clinical Orthopaedics and Related Research® / Issue 9/2011
Print ISSN: 0009-921X
Electronic ISSN: 1528-1132
DOI
https://doi.org/10.1007/s11999-011-1773-6

Other articles of this Issue 9/2011

Clinical Orthopaedics and Related Research® 9/2011 Go to the issue

Symposium: Reverse Total Shoulder Arthroplasty

A Radiographic Classification of Massive Rotator Cuff Tear Arthritis

Symposium: Reverse Total Shoulder Arthroplasty

The Classic: Articular Replacement for the Humeral Head