Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Knee Surgery, Sports Traumatology, Arthroscopy
Published in: Knee Surgery, Sports Traumatology, Arthroscopy 3/2023

Open Access 11-11-2022 | Knee Osteoarthritis | KNEE

Artificial intelligence-based computer-aided system for knee osteoarthritis assessment increases experienced orthopaedic surgeons’ agreement rate and accuracy

Authors: Maria Anna Smolle, Christoph Goetz, Dietmar Maurer, Ines Vielgut, Michael Novak, Gerhard Zier, Andreas Leithner, Stefan Nehrer, Tiago Paixao, Richard Ljuhar, Patrick Sadoghi

Published in: Knee Surgery, Sports Traumatology, Arthroscopy | Issue 3/2023

Login to get access

Abstract

Purpose

The aims of this study were to (1) analyze the impact of an artificial intelligence (AI)-based computer system on the accuracy and agreement rate of board-certified orthopaedic surgeons (= senior readers) to detect X-ray features indicative of knee OA in comparison to unaided assessment and (2) compare the results to those of senior residents (= junior readers).

Methods

One hundred and twenty-four unilateral knee X-rays from the OAI study were analyzed regarding Kellgren–Lawrence grade, joint space narrowing (JSN), sclerosis and osteophyte OARSI grade by computerized methods. Images were rated for these parameters by three senior readers using two modalities: plain X-ray (unaided) and X-ray presented alongside reports from a computer-assisted detection system (aided). After exclusion of nine images with incomplete annotation, intraclass correlations between readers were calculated for both modalities among 115 images, and reader performance was compared to ground truth (OAI consensus). Accuracy, sensitivity and specificity were also calculated and the results were compared to those from a previous study on junior readers.

Results

With the aided modality, senior reader agreement rates for KL grade (2.0-fold), sclerosis (1.42-fold), JSN (1.37-fold) and osteophyte OARSI grades (3.33-fold) improved significantly. Reader specificity and accuracy increased significantly for all features when using the aided modality compared to the gold standard. On the other hand, sensitivity only increased for OA diagnosis, whereas it decreased (without statistical significance) for all other features. With aided analysis, senior readers reached similar agreement and accuracy rates as junior readers, with both surpassing AI performance.

Conclusion

The introduction of AI-based computer-aided assessment systems can increase the agreement rate and overall accuracy for knee OA diagnosis among board-certified orthopaedic surgeons. Thus, use of this software may improve the standard of care for knee OA detection and diagnosis in the future.

Level of evidence

Level II.
Literature
1.
Feigin V (2016) Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 388:1545–1602CrossRef
2.
Cui A, Li H, Wang D, Zhong J, Chen Y, Lu H (2020) Global, regional prevalence, incidence and risk factors of knee osteoarthritis in population-based studies. EClinicalMedicine 29–30:100587CrossRefPubMedPubMedCentral
3.
Litwic A, Edwards MH, Dennison EM, Cooper C (2013) Epidemiology and burden of osteoarthritis. Br Med Bull 105:185–199CrossRefPubMed
4.
5.
Spector TD, Cooper C (1993) Radiographic assessment of osteoarthritis in population studies: whither Kellgren and Lawrence? Osteoarthr Cartil 1:203–206CrossRef
6.
Schiphof D, de Klerk BM, Kerkhof HJ, Hofman A, Koes BW, Boers M et al (2011) Impact of different descriptions of the Kellgren and Lawrence classification criteria on the diagnosis of knee osteoarthritis. Ann Rheum Dis 70:1422–1427CrossRefPubMed
7.
Altman RD, Gold GE (2007) Atlas of individual radiographic features in osteoarthritis, revised. Osteoarthr Cartil 15(Suppl A):A1–A56CrossRef
8.
Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S (2018) Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 8:1727CrossRefPubMedPubMedCentral
9.
Culvenor AG, Engen CN, Oiestad BE, Engebretsen L, Risberg MA (2015) Defining the presence of radiographic knee osteoarthritis: a comparison between the Kellgren and Lawrence system and OARSI atlas criteria. Knee Surg Sports Traumatol Arthrosc 23:3532–3539CrossRefPubMed
10.
Gossec L, Jordan JM, Mazzuca SA, Lam MA, Suarez-Almazor ME, Renner JB et al (2008) Comparative evaluation of three semi-quantitative radiographic grading techniques for knee osteoarthritis in terms of validity and reproducibility in 1759 X-rays: report of the OARSI-OMERACT task force. Osteoarthr Cartil 16:742–748CrossRef
11.
Sheehy L, Culham E, McLean L, Niu J, Lynch J, Segal NA et al (2015) Validity and sensitivity to change of three scales for the radiographic assessment of knee osteoarthritis using images from the Multicenter Osteoarthritis Study (MOST). Osteoarthr Cartil 23:1491–1498CrossRef
12.
Damen J, Schiphof D, Wolde ST, Cats HA, Bierma-Zeinstra SM, Oei EH (2014) Inter-observer reliability for radiographic assessment of early osteoarthritis features: the CHECK (cohort hip and cohort knee) study. Osteoarthr Cartil 22:969–974CrossRef
13.
Kinds MB, Welsing PM, Vignon EP, Bijlsma JW, Viergever MA, Marijnissen AC et al (2011) A systematic review of the association between radiographic and clinical osteoarthritis of hip and knee. Osteoarthr Cartil 19:768–778CrossRef
14.
Schiphof D, Boers M, Bierma-Zeinstra SM (2008) Differences in descriptions of Kellgren and Lawrence grades of knee osteoarthritis. Ann Rheum Dis 67:1034–1036CrossRefPubMed
15.
Chung SW, Han SS, Lee JW, Oh KS, Kim NR, Yoon JP et al (2018) Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 89:468–473CrossRefPubMedPubMedCentral
16.
Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A et al (2017) Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop 88:581–586CrossRefPubMedPubMedCentral
17.
18.
van Leeuwen KG, Schalekamp S, Rutten M, van Ginneken B, de Rooij M (2021) Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur Radiol 31:3797–3804CrossRefPubMedPubMedCentral
19.
Nehrer S, Ljuhar R, Steindl P, Simon R, Maurer D, Ljuhar D et al (2021) Automated knee osteoarthritis assessment increases physicians’ agreement rate and accuracy: data from the osteoarthritis initiative. Cartilage 13:957s–965sCrossRefPubMed
20.
21.
Brejnebøl MW, Hansen P, Nybing JU, Bachmann R, Ratjen U, Hansen IV et al (2022) External validation of an artificial intelligence tool for radiographic knee osteoarthritis severity classification. Eur J Radiol 150:110249CrossRefPubMed
22.
23.
Kothari M, Guermazi A, von Ingersleben G, Miaux Y, Sieffert M, Block JE et al (2004) Fixed-flexion radiography of the knee provides reproducible joint space width measurements in osteoarthritis. Eur Radiol 14:1568–1573CrossRefPubMed
24.
Niinimäki T, Ojala R, Niinimäki J, Leppilahti J (2010) The standing fixed flexion view detects narrowing of the joint space better than the standing extended view in patients with moderate osteoarthritis of the knee. Acta Orthop 81:344–346CrossRefPubMedPubMedCentral
25.
Peterfy C, Li J, Zaim S, Duryea J, Lynch J, Miaux Y et al (2003) Comparison of fixed-flexion positioning with fluoroscopic semi-flexed positioning for quantifying radiographic joint-space width in the knee: test-retest reproducibility. Skeletal Radiol 32:128–132CrossRefPubMed
26.
27.
Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428CrossRefPubMed
28.
Cicchetti DV (1994) Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology, vol 6. American Psychological Association, US, pp 284–290
29.
Guermazi A, Hunter DJ, Li L, Benichou O, Eckstein F, Kwoh CK et al (2012) Different thresholds for detecting osteophytes and joint space narrowing exist between the site investigators and the centralized reader in a multicenter knee osteoarthritis study–data from the Osteoarthritis Initiative. Skeletal Radiol 41:179–186CrossRefPubMed
30.
Felson DT, Niu J, Guermazi A, Sack B, Aliabadi P (2011) Defining radiographic incidence and progression of knee osteoarthritis: suggested modifications of the Kellgren and Lawrence scale. Ann Rheum Dis 70:1884–1886CrossRefPubMed
31.
Madry H, Kon E, Condello V, Peretti GM, Steinwachs M, Seil R et al (2016) Early osteoarthritis of the knee. Knee Surg Sports Traumatol Arthrosc 24:1753–1762CrossRefPubMed
32.
Günther KP, Sun Y (1999) Reliability of radiographic assessment in hip and knee osteoarthritis. Osteoarthr Cartil 7:239–246CrossRef
33.
Peterlein CD, Schuttler KF, Lakemeier S, Timmesfeld N, Gorg C, Fuchs-Winkelmann S et al (2010) Reproducibility of different screening classifications in ultrasonography of the newborn hip. BMC Pediatr 10:98CrossRefPubMedPubMedCentral
34.
Tversky A, Kahneman D (1974) Judgment under uncertainty: heuristics and biases. Science 185:1124–1131CrossRefPubMed
35.
Sherif M, Taub D, Hovland CI (1958) Assimilation and contrast effects of anchoring stimuli on judgments. J Exp Psychol 55:150–155CrossRefPubMed
36.
Zhang Y, Lewis M, Pellon M, Coleman P. A preliminary research on modeling cognitive agents for social environments in multi-agent systems. 2007.
37.
Hardesty LA, Ganott MA, Hakim CM, Cohen CS, Clearfield RJ, Gur D (2005) “Memory effect” in observer performance studies of mammograms. Acad Radiol 12:286–290CrossRefPubMed
38.
Ryan JT, Haygood TM, Yamal JM, Evanoff M, O’Sullivan P, McEntee M et al (2011) The “memory effect” for repeated radiologic observations. AJR Am J Roentgenol 197:W985-991CrossRefPubMed
Metadata
Title
Artificial intelligence-based computer-aided system for knee osteoarthritis assessment increases experienced orthopaedic surgeons’ agreement rate and accuracy
Authors
Maria Anna Smolle
Christoph Goetz
Dietmar Maurer
Ines Vielgut
Michael Novak
Gerhard Zier
Andreas Leithner
Stefan Nehrer
Tiago Paixao
Richard Ljuhar
Patrick Sadoghi
Publication date
11-11-2022
Publisher
Springer Berlin Heidelberg
Published in
Knee Surgery, Sports Traumatology, Arthroscopy / Issue 3/2023
Print ISSN: 0942-2056
Electronic ISSN: 1433-7347
DOI
https://doi.org/10.1007/s00167-022-07220-y

Other articles of this Issue 3/2023

Knee Surgery, Sports Traumatology, Arthroscopy 3/2023 Go to the issue

KNEE

Metaphyseal cones and sleeves are similar in improving short- and mid-term outcomes in Total Knee Arthroplasty revisions

Knee

A BMI above 30 results in satisfying outcomes in patients undergoing fixed-bearing lateral unicompartmental knee arthroplasty

KNEE

Total knee arthroplasty improves sports activity and the patient-reported functional outcome at mid-term follow-up

KNEE

Successful same-day discharge in 88% of patients after unicompartmental knee arthroplasty: a systematic review and meta-analysis

KNEE

Highly cross-linked polyethylene versus conventional polyethylene in primary total knee arthroplasty: comparable clinical and radiological results at a 10-year follow-up

KNEE

No advantage with navigated versus conventional mechanically aligned total knee arthroplasty—10 year results of a randomised controlled trial