Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Clinical Orthopaedics and Related Research®
Published in: Clinical Orthopaedics and Related Research® 3/2017

01-03-2017 | Symposium: 2015 Meetings of the Musculoskeletal Tumor Society and the International Society of Limb Salvage

Has the Level of Evidence of Podium Presentations at the Musculoskeletal Tumor Society Annual Meeting Changed Over Time?

Authors: Daniel M. Lerman, MD, Matthew G. Cable, MD, Patrick Thornley, BHSc, Nathan Evaniew, MD, Gerard P. Slobogean, MD, Mohit Bhandari, MD, MSc, PhD, John H. Healey, MD, R. Lor Randall, MD, Michelle Ghert, MD

Published in: Clinical Orthopaedics and Related Research® | Issue 3/2017

Login to get access

Abstract

Background

Level of evidence (LOE) framework is a tool with which to categorize clinical studies based on their likelihood to be influenced by bias. Improvements in LOE have been demonstrated throughout orthopaedics, prompting our evaluation of orthopaedic oncology research LOE to determine if it has changed in kind.

Questions/purposes

(1) Has the LOE presented at the Musculoskeletal Tumor Society (MSTS) annual meeting improved over time? (2) Over the past decade, how do the MSTS and Orthopaedic Trauma Association (OTA) annual meetings compare regarding LOE overall and for the subset of therapeutic studies?

Methods

We reviewed abstracts from MSTS and OTA annual meeting podium presentations from 2005 to 2014. Three independent reviewers evaluated a total of 1222 abstracts for study type and LOE; there were 577 abstracts from MSTS and 645 from OTA. Changes in the distributions of study type and LOE over time were evaluated by Pearson chi-square test.

Results

There was no change over time in MSTS LOE for all study types (p = 0.13) and therapeutic (p = 0.36) study types during the reviewed decade. In contrast, OTA LOE increased over this time for all study types (p < 0.01). The proportion of Level I therapeutic studies was higher at the OTA than the MSTS (3% [14 of 413] versus 0.5% [two of 387], respectively), whereas the proportion of Level IV studies was lower at the OTA than the MSTS (32% [134 of 413] versus 75% [292 of 387], respectively) during the reviewed decade. The proportion of controlled therapeutic studies (LOE I through III) versus uncontrolled studies (LOE IV) increased over time at OTA (p < 0.021), but not at MSTS (p = 0.10).

Conclusions

Uncontrolled case series continue to dominate the MSTS scientific program, limiting progress in evidence-based clinical care. Techniques used by the OTA to improve LOE may be emulated by the MSTS. These techniques focus on broad participation in multicenter collaborations that are designed in a comprehensive manner and answer a pragmatic clinical question.
Literature
1.
Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64:401–406.CrossRefPubMed
2.
3.
4.
Bhandari M, Swiontkowski MF, Einhorn TA, Tornetta P 3rd, Schemitsch EH, Leece P, Sprague S, Wright JG. Interobserver agreement in the application of levels of evidence to scientific papers in the American volume of the Journal of Bone and Joint Surgery. J Bone Joint Surg Am. 2004;86:1717–1720.CrossRefPubMed
5.
Bielack SS, Smeland S, Whelan JS, Marina N, Jovic G, Hook JM, Krailo MD, Gebhardt M, Papai Z, Meyer J, Nadel H, Randall RL, Deffenbaugh C, Nagarajan R, Brennan B, Letson GD, Teot LA, Goorin A, Baumhoer D, Kager L, Werner M, Lau CC, Sundby Hall K, Gelderblom H, Meyers P, Gorlick R, Windhager R, Helmke K, Eriksson M, Hoogerbrugge PM, Schomberg P, Tunn PU, Kuhne T, Jurgens H, van den Berg H, Bohling T, Picton S, Renard M, Reichardt P, Gerss J, Butterfass-Bahloul T, Morris C, Hogendoorn PC, Seddon B, Calaminus G, Michelagnoli M, Dhooge C, Sydes MR, Bernstein M, EURAMOS-1 investigators. Methotrexate, foxorubicin, and visplatin (MAP) Plus maintenance pegylated interferon alfa-2b versus MAP alone in patients with resectable high-grade osteosarcoma and good histologic response to preoperative MAP: first results of the EURAMOS-1 good response randomized controlled trial. J Clin Oncol. 2015;33:2279–2287.
6.
Bogaerts J, Sydes MR, Keat N, McConnell A, Benson A, Ho A, Roth A, Fortpied C, Eng C, Peckitt C, Coens C, Pettaway C, Arnold D, Hall E, Marshall E, Sclafani F, Hatcher H, Earl H, Ray-Coquard I, Paul J, Blay JY, Whelan J, Panageas K, Wheatley K, Harrington K, Licitra L, Billingham L, Hensley M, McCabe M, Patel PM, Carvajal R, Wilson R, Glynne-Jones R, McWilliams R, Leyvraz S, Rao S, Nicholson S, Filiaci V, Negrouk A, Lacombe D, Dupont E, Pauporte I, Welch JJ, Law K, Trimble T, Seymour M. Clinical trial designs for rare diseases: studies developed and discussed by the International Rare Cancers Initiative. Eur J Cancer. 2015;51:271–281.CrossRefPubMedPubMedCentral
7.
Bone LB, Johnson KD, Weigelt J, Scheinberg R. Early versus delayed stabilization of femoral fractures. A prospective randomized study. J Bone Joint Surg Am. 1989;71:336–340.CrossRefPubMed
8.
Buckley R, Leighton R, Sanders D, Poon J, Coles CP, Stephen D, Paolucci EO. Open reduction and internal fixation compared with ORIF and primary subtalar arthrodesis for treatment of Sanders type IV calcaneal fractures: a randomized multicenter trial. J Orthop Trauma. 2014;28:577–583.CrossRefPubMed
9.
Buckley R, Tough S, McCormack R, Pate G, Leighton R, Petrie D, Galpin R. Operative compared with nonoperative treatment of displaced intra-articular calcaneal fractures: a prospective, randomized, controlled multicenter trial. J Bone Joint Surg Am. 2002;84:1733–1744.CrossRefPubMed
10.
Cashin MS, Kelley SP, Douziech JR, Varghese RA, Hamilton QP, Mulpuri K. The levels of evidence in pediatric orthopaedic journals: where are we now? J Pediatr Orthop. 2011;31:721–725.CrossRefPubMed
11.
Castillo RC, Scharfstein DO, MacKenzie EJ. Observational studies in the era of randomized trials: finding the balance. J Bone Joint Surg Am. 2012;94(Suppl 1):112–117.CrossRefPubMed
12.
Cunningham BP, Harmsen S, Kweon C, Patterson J, Waldrop R, McLaren A, McLemore R. Have levels of evidence improved the quality of orthopaedic research? Clin Orthop Relat Res. 2013;471:3679–3686.CrossRefPubMedPubMedCentral
13.
Devereaux PJ, Bhandari M, Clarke M, Montori VM, Cook DJ, Yusuf S, Sackett DL, Cina CS, Walter SD, Haynes B, Schunemann HJ, Norman GR, Guyatt GH. Need for expertise based randomised controlled trials. BMJ. 2005;330:88.CrossRefPubMedPubMedCentral
14.
Dowrick AS, Bhandari M. Ethical issues in the design of randomized trials: to sham or not to sham. J Bone Joint Surg Am. 2012;94(Suppl 1):7–10.CrossRefPubMed
15.
Evaniew N, Nuttall J, Farrokhyar F, Bhandari M, Ghert M. What are the levels of evidence on which we base decisions for surgical management of lower extremity bone tumors? Clin Orthop Relat Res. 2014;472:8–15.CrossRefPubMed
16.
Ghert M, Deheshi B, Holt G, Randall RL, Ferguson P, Wunder J, Turcotte R, Werier J, Clarkson P, Damron T, Benevenia J, Anderson M, Gebhardt M, Isler M, Mottard S, Healey J, Evaniew N, Racano A, Sprague S, Swinton M, Bryant D, Thabane L, Guyatt G, Bhandari M, PARITY Investigators. Prophylactic antibiotic regimens in tumour surgery (PARITY): protocol for a multicentre randomised controlled study. BMJ Open. 2012;2.
17.
Ghert M, Petrisor B. Subgroup analyses: when should we believe them? J Bone Joint Surg Am. 2012;94(Suppl 1):61–64.CrossRefPubMed
18.
Grant HM, Tjoumakaris FP, Maltenfort MG, Freedman KB. Levels of evidence in the clinical sports medicine literature: are we getting better over time? Am J Sports Med. 2014;42:1738–1742.CrossRefPubMed
19.
Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schunemann HJ. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64:383–394.CrossRefPubMed
20.
21.
Hanzlik S, Mahabir RC, Baynosa RC, Khiabani KT. Levels of evidence in research published in The Journal of Bone and Joint Surgery (American Volume) over the last thirty years. J Bone Joint Surg Am. 2009;91:425–428.CrossRefPubMed
22.
Hoppe DJ, Schemitsch EH, Morshed S, Tornetta P 3rd, Bhandari M. Hierarchy of evidence: where observational studies fit in and why we need them. J Bone Joint Surg Am. 2009;91(Suppl 3):2–9.CrossRefPubMed
23.
Jasko JJ, Wood JH, Schwartz HS. Publication rates of abstracts presented at annual Musculoskeletal Tumor Society meetings. Clin Orthop Relat Res. 2003;415:98–103.CrossRef
24.
Karanicolas PJ, Bhandari M, Taromi B, Akl EA, Bassler D, Alonso-Coello P, Rigau D, Bryant D, Smith SE, Walter SD, Guyatt GH. Blinding of outcomes in trials of orthopaedic trauma: an opportunity to enhance the validity of clinical trials. J Bone Joint Surg Am. 2008;90:1026–1033.CrossRefPubMed
25.
Khan H, Hussain N, Bhandari M. The influence of large clinical trials in orthopedic trauma: do they change practice? J Orthop Trauma. 2013;27:e268–274.
26.
Kooistra B, Dijkman B, Einhorn TA, Bhandari M. How to design a good case series. J Bone Joint Surg Am. 2009;91(Suppl 3):21–26.CrossRefPubMed
27.
Moseley JB, O’Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, Hollingsworth JC, Ashton CM, Wray NP. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002;347:81–88.CrossRefPubMed
28.
29.
Murphy RF, Cibulas AM, Sawyer JR, Spence DD, Kelly DM. Levels of Evidence in the Journal of Pediatric Orthopaedics: update and comparison to the Journal of Bone and Joint Surgery. J Pediatr Orthop. 2015;35:779–781.CrossRefPubMed
30.
Nazarian A, Entezari V, Zurakowski D, Calderon N, Hipp JA, Villa-Camacho JC, Lin PP, Cheung FH, Aboulafia AJ, Turcotte R, Anderson ME, Gebhardt MC, Cheng EY, Terek RM, Yaszemski M, Damron TA, Snyder BD. Treatment planning and fracture prediction in patients with skeletal metastasis with CT-based rigidity analysis. Clin Cancer Res. 2015;21:2514–2519.CrossRefPubMedPubMedCentral
31.
Ng VY, Scharschmidt TJ, Mayerson JL, Fisher JL. Incidence and survival in sarcoma in the United States: a focus on musculoskeletal lesions. Anticancer Res. 2013;33:2597–2604.PubMed
32.
O’Sullivan B, Davis AM, Turcotte R, Bell R, Catton C, Chabot P, Wunder J, Kandel R, Goddard K, Sadura A, Pater J, Zee B. Preoperative versus postoperative radiotherapy in soft-tissue sarcoma of the limbs: a randomised trial. Lancet. 2002;359:2235–2241.CrossRefPubMed
33.
Obremskey WT, Pappas N, Attallah-Wasif E, Tornetta P 3rd, Bhandari M. Level of evidence in orthopaedic journals. J Bone Joint Surg Am. 2005;87:2632–2638.PubMed
34.
35.
36.
Pearson AM, Tosteson AN, Koval KJ, McKee MD, Cantu RV, Bell JE, Vicente M. Is surgery for displaced, midshaft clavicle fractures in adults cost-effective? Results based on a multicenter randomized, controlled trial. J Orthop Trauma. 2010;24:426–433.CrossRefPubMed
37.
Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1986;89:2S–3S.CrossRefPubMed
38.
39.
Sibai T, Carlisle H, Tornetta P 3rd. The darker side of randomized trials: recruitment challenges. J Bone Joint Surg Am. 2012;94(Suppl 1):49–55.CrossRefPubMed
40.
Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85:257–268.PubMed
41.
Slobogean GP, Dielwart C, Johal HS, Shantz JA, Mulpuri K. Levels of evidence at the Orthopaedic Trauma Association annual meetings. J Orthop Trauma. 2013;27:e208–212.
42.
Slobogean GP, Sprague S, Bhandari M. The tactics of large randomized trials. J Bone Joint Surg Am. 2012;94(Suppl 1):19–23.CrossRefPubMed
43.
Sprague S, Bhandari M, Devereaux PJ, Swiontkowski MF, Tornetta P 3rd, Cook DJ, Dirschl D, Schemitsch EH, Guyatt GH. Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. J Bone Joint Surg Am. 2003;85:158–163.CrossRefPubMed
44.
Study to Prospectively Evaluate Reamed Intramedullary Nails in Patients with Tibial Fractures Investigators, Bhandari M, Guyatt G, Tornetta P 3rd, Schemitsch EH, Swiontkowski M, Sanders D, Walter SD. Randomized trial of reamed and unreamed intramedullary nailing of tibial shaft fractures. J Bone Joint Surg Am. 2008;90:2567–2578.
45.
Tornetta P 3rd, Pascual M, Godin K, Sprague S, Bhandari M. Participating in multicenter randomized controlled trials: what’s the relative value? J Bone Joint Surg Am. 2012;94(Suppl 1):107–111.CrossRefPubMed
46.
Voleti PB, Donegan DJ, Baldwin KD, Lee GC. Level of evidence of presentations at American Academy of Orthopaedic Surgeons annual meetings. J Bone Joint Surg Am. 2012;94:e50.CrossRefPubMed
47.
Whelan JS, Bielack SS, Marina N, Smeland S, Jovic G, Hook JM, Krailo M, Anninga J, Butterfass-Bahloul T, Bohling T, Calaminus G, Capra M, Deffenbaugh C, Dhooge C, Eriksson M, Flanagan AM, Gelderblom H, Goorin A, Gorlick R, Gosheger G, Grimer RJ, Hall KS, Helmke K, Hogendoorn PC, Jundt G, Kager L, Kuehne T, Lau CC, Letson GD, Meyer J, Meyers PA, Morris C, Mottl H, Nadel H, Nagarajan R, Randall RL, Schomberg P, Schwarz R, Teot LA, Sydes MR, Bernstein M, EURAMOS-1 collaborators. EURAMOS-1, an international randomised study for osteosarcoma: results from pre-randomisation treatment. Ann Oncol. 2015;26:407–414.
48.
Womer RB, West DC, Krailo MD, Dickman PS, Pawel BR, Grier HE, Marcus K, Sailer S, Healey JH, Dormans JP, Weiss AR. Randomized controlled trial of interval-compressed chemotherapy for the treatment of localized Ewing sarcoma: a report from the Children’s Oncology Group. J Clin Oncol. 2012;30:4148–4154.CrossRefPubMedPubMedCentral
49.
Wright JG, Swiontkowski MF, Heckman JD. Introducing levels of evidence to the journal. J Bone Joint Surg Am. 2003;85:1–3.CrossRefPubMed
50.
Zaidi R, Abbassian A, Cro S, Guha A, Cullen N, Singh D, Goldberg A. Levels of evidence in foot and ankle surgery literature: progress from 2000 to 2010? J Bone Joint Surg Am. 2012;94:e1121–1110.
Metadata
Title
Has the Level of Evidence of Podium Presentations at the Musculoskeletal Tumor Society Annual Meeting Changed Over Time?
Authors
Daniel M. Lerman, MD
Matthew G. Cable, MD
Patrick Thornley, BHSc
Nathan Evaniew, MD
Gerard P. Slobogean, MD
Mohit Bhandari, MD, MSc, PhD
John H. Healey, MD
R. Lor Randall, MD
Michelle Ghert, MD
Publication date
01-03-2017
Publisher
Springer US
Published in
Clinical Orthopaedics and Related Research® / Issue 3/2017
Print ISSN: 0009-921X
Electronic ISSN: 1528-1132
DOI
https://doi.org/10.1007/s11999-016-4763-x

Other articles of this Issue 3/2017

Clinical Orthopaedics and Related Research® 3/2017 Go to the issue

Erratum

Erratum to: What Is the Role of Allogeneic Cortical Strut Grafts in the Treatment of Fibrous Dysplasia of the Proximal Femur?

CORR Insights

CORR Insights®: Hypotensive Epidural Anesthesia Reduces Blood Loss in Pelvic and Sacral Bone Tumor Resections

Cochrane in CORR

Cochrane in CORR ®: Intramedullary Nailing for Tibial Shaft Fractures in Adults (Review)

Symposium: 2015 Meetings of the Musculoskeletal Tumor Society and the International Society of Limb Salvage

Proximal Tibia Reconstruction After Bone Tumor Resection: Are Survivorship and Outcomes of Endoprosthetic Replacement and Osteoarticular Allograft Similar?

Symposium: 2015 Meetings of the Musculoskeletal Tumor Society and the International Society of Limb Salvage

Is It Possible and Safe to Perform Acetabular-preserving Resections for Malignant Neoplasms of the Periacetabular Region?

Symposium: 2015 Meetings of the Musculoskeletal Tumor Society and the International Society of Limb Salvage

What Sport Activity Levels Are Achieved in Patients After Resection and Endoprosthetic Reconstruction for a Proximal Femur Bone Sarcoma?