Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Current Reviews in Musculoskeletal Medicine
Published in: Current Reviews in Musculoskeletal Medicine 6/2024

08-04-2024 | Artificial Intelligence | REVIEW

Artificial Intelligence for Clinically Meaningful Outcome Prediction in Orthopedic Research: Current Applications and Limitations

Authors: Seong Jun Jang, Jake Rosenstadt, Eugenia Lee, Kyle N. Kunze

Published in: Current Reviews in Musculoskeletal Medicine | Issue 6/2024

Login to get access

Abstract

Purpose of Review

Patient-reported outcome measures (PROM) play a critical role in evaluating the success of treatment interventions for musculoskeletal conditions. However, predicting which patients will benefit from treatment interventions is complex and influenced by a multitude of factors. Artificial intelligence (AI) may better anticipate the propensity to achieve clinically meaningful outcomes through leveraging complex predictive analytics that allow for personalized medicine. This article provides a contemporary review of current applications of AI developed to predict clinically significant outcome (CSO) achievement after musculoskeletal treatment interventions.

Recent Findings

The highest volume of literature exists in the subspecialties of total joint arthroplasty, spine, and sports medicine, with only three studies identified in the remaining orthopedic subspecialties combined. Performance is widely variable across models, with most studies only reporting discrimination as a performance metric. Given the complexity inherent in predictive modeling for this task, including data availability, data handling, model architecture, and outcome selection, studies vary widely in their methodology and results. Importantly, the majority of studies have not been externally validated or demonstrate important methodological limitations, precluding their implementation into clinical settings.

Summary

A substantial body of literature has accumulated demonstrating variable internal validity, limited scope, and low potential for clinical deployment. The majority of studies attempt to predict the MCID—the lowest bar of clinical achievement. Though a small proportion of models demonstrate promise and highlight the utility of AI, important methodological limitations need to be addressed moving forward to leverage AI-based applications for clinical deployment.
Literature
1.
Pasqualini I, Piuzzi NS. New CMS policy on the mandatory collection of patient-reported outcome measures for total hip and knee arthroplasty by 2027: what orthopaedic surgeons should know. J Bone Joint Surg Am. 2024.
2.
Makhni EC. Meaningful clinical applications of patient-reported outcome measures in orthopaedics. J Bone Joint Surg Am. 2021;103(1):84–91.PubMedCrossRef
3.
4.
Makhni EC, Baumhauer JF, Ayers D, Bozic KJ. Patient-reported outcome measures: how and why they are collected. Instr Course Lect. 2019;68:675–80.PubMed
5.
Chung AS, Copay AG, Olmscheid N, Campbell D, Walker JB, Chutkan N. Minimum Clinically important difference: current trends in the spine literature. Spine (Phila Pa 1976). 2017;42(14):1096–105.
6.
Copay AG, Chung AS, Eyberg B, Olmscheid N, Chutkan N, Spangehl MJ. Minimum clinically important difference: current trends in the orthopaedic literature, Part I: Upper Extremity: A Systematic Review. JBJS Rev. 2018;6(9): e1.PubMedCrossRef
7.
Copay AG, Eyberg B, Chung AS, Zurcher KS, Chutkan N, Spangehl MJ. Minimum clinically important difference: current trends in the orthopaedic literature, Part II: lower extremity: a systematic review. JBJS Rev. 2018;6(9): e2.PubMedCrossRef
8.
9.
Kunze KN, Bart JA, Ahmad M, Nho SJ, Chahla J. Large heterogeneity among minimal clinically important differences for hip arthroscopy outcomes: a systematic review of reporting trends and quantification methods. Arthroscopy. 2021;37(3):1028-37 e6.PubMedCrossRef
10.
Menendez ME, Sudah SY, Cohn MR, Narbona P, Ladermann A, Barth J, et al. Defining minimal clinically important difference and patient acceptable symptom state after the latarjet procedure. Am J Sports Med. 2022;50(10):2761–6.PubMedCrossRef
11.
Bernstein DN, Karhade AV, Bono CM, Schwab JH, Harris MB, Tobert DG. Sociodemographic factors are associated with patient-reported outcome measure completion in orthopaedic surgery: an analysis of completion rates and determinants among new patients. JB JS Open Access. 2022;7(3):e22.00026.
12.
Jolback P, Rolfson O, Mohaddes M, Nemes S, Karrholm J, Garellick G, et al. Does surgeon experience affect patient-reported outcomes 1 year after primary total hip arthroplasty? Acta Orthop. 2018;89(3):265–71.PubMedPubMedCentralCrossRef
13.
Langlotz CP, Allen B, Erickson BJ, Kalpathy-Cramer J, Bigelow K, Cook TS, et al. A roadmap for foundational research on artificial intelligence in medical imaging: from the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology. 2019;291(3):781–91.PubMedCrossRef
14.
Padash S, Mickley JP, Vera Garcia DV, Nugen F, Khosravi B, Erickson BJ, et al. An overview of machine learning in orthopedic surgery: an educational paper. J Arthroplasty. 2023;38(10):1938–42.PubMedCrossRef
15.
Langenberger B, Schrednitzki D, Halder AM, Busse R, Pross CM. Predicting whether patients will achieve minimal clinically important differences following hip or knee arthroplasty. Bone Joint Res. 2023;12(9):512–21.PubMedPubMedCentralCrossRef
16.
Fontana MA, Lyman S, Sarker GK, Padgett DE, MacLean CH. Can machine learning algorithms predict which patients will achieve minimally clinically important differences from total joint arthroplasty? Clin Orthop Relat Res. 2019;477(6):1267–79.PubMedPubMedCentralCrossRef
17.
Nwachukwu BU, Beck EC, Lee EK, Cancienne JM, Waterman BR, Paul K, et al. Application of machine learning for predicting clinically meaningful outcome after arthroscopic femoroacetabular impingement surgery. Am J Sports Med. 2020;48(2):415–23.PubMedCrossRef
18.
Kunze KN, Krivicich LM, Clapp IM, Bodendorfer BM, Nwachukwu BU, Chahla J, et al. Machine learning algorithms predict achievement of clinically significant outcomes after orthopaedic surgery: a systematic review. Arthroscopy. 2022;38(6):2090–105.PubMedCrossRef
19.
El-Othmani MM, Zalikha AK, Shah RP. Comparative analysis of the ability of machine learning models in predicting in-hospital postoperative outcomes after total hip arthroplasty. J Am Acad Orthop Surg. 2022;30(20):e1337–47.PubMedCrossRef
20.
Rouzrokh P, Mickley JP, Khosravi B, Faghani S, Moassefi M, Schulz WR, et al. THA-AID: deep learning tool for total hip arthroplasty automatic implant detection with uncertainty and outlier quantification. J Arthroplasty. 2023;9(4):966–73.
21.
Khosravi B, Rouzrokh P, Mickley JP, Faghani S, Larson AN, Garner HW, et al. Creating high fidelity synthetic pelvis radiographs using generative adversarial networks: unlocking the potential of deep learning models without patient privacy concerns. J Arthroplasty. 2023;38(10):2037-43 e1.PubMedCrossRef
22.
Jang SJ, Fontana MA, Kunze KN, Anderson CG, Sculco TP, Mayman DJ, et al. An interpretable machine learning model for predicting 10-year total hip arthroplasty risk. J Arthroplasty. 2023;38(7S):S44–50.
23.
Huber M, Kurz C, Leidl R. Predicting patient-reported outcomes following hip and knee replacement surgery using supervised machine learning. BMC Med Inform Decis Mak. 2019;19(1):3.PubMedPubMedCentralCrossRef
24.
Harris AHS, Kuo AC, Bowe TR, Manfredi L, Lalani NF, Giori NJ. Can machine learning methods produce accurate and easy-to-use preoperative prediction models of one-year improvements in pain and functioning after knee arthroplasty? J Arthroplasty. 2021;36(1):112-7 e6.PubMedCrossRef
25.
Katakam A, Karhade AV, Collins A, Shin D, Bragdon C, Chen AF, et al. Development of machine learning algorithms to predict achievement of minimal clinically important difference for the KOOS-PS following total knee arthroplasty. J Orthop Res. 2022;40(4):808–15.PubMedCrossRef
26.
Zhang S, Lau BPH, Ng YH, Wang X, Chua W. Machine learning algorithms do not outperform preoperative thresholds in predicting clinically meaningful improvements after total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc. 2022;30(8):2624–30.PubMedCrossRef
27.
Kunze KN, Karhade AV, Sadauskas AJ, Schwab JH, Levine BR. Development of machine learning algorithms to predict clinically meaningful improvement for the patient-reported health state after total hip arthroplasty. J Arthroplasty. 2020;35(8):2119–23.PubMedCrossRef
28.
Bourne RB, Chesworth BM, Davis AM, Mahomed NN, Charron KD. Patient satisfaction after total knee arthroplasty: who is satisfied and who is not? Clin Orthop Relat Res. 2010;468(1):57–63.PubMedCrossRef
29.
Farooq H, Deckard ER, Ziemba-Davis M, Madsen A, Meneghini RM. Predictors of patient satisfaction following primary total knee arthroplasty: results from a traditional statistical model and a machine learning algorithm. J Arthroplasty. 2020;35(11):3123–30.PubMedCrossRef
30.
Pettit MH, Hickman SHM, Malviya A, Khanduja V. Development of machine-learning algorithms to predict attainment of minimal clinically important difference after hip arthroscopy for femoroacetabular impingement yield fair performance and limited clinical utility. Arthroscopy. 2023;40(4):1153–63.
31.
Ramkumar PN, Karnuta JM, Haeberle HS, Sullivan SW, Nawabi DH, Ranawat AS, et al. Radiographic indices are not predictive of clinical outcomes among 1735 patients indicated for hip arthroscopic surgery: a machine learning analysis. Am J Sports Med. 2020;48(12):2910–8.PubMedCrossRef
32.
Kunze KN, Polce EM, Rasio J, Nho SJ. Machine learning algorithms predict clinically significant improvements in satisfaction after hip arthroscopy. Arthroscopy. 2021;37(4):1143–51.PubMedCrossRef
33.
Kunze KN, Polce EM, Nwachukwu BU, Chahla J, Nho SJ. Development and internal validation of supervised machine learning algorithms for predicting clinically significant functional improvement in a mixed population of primary hip arthroscopy. Arthroscopy. 2021;37(5):1488–97.PubMedCrossRef
34.
•Kunze KN, Polce EM, Clapp IM, Alter T, Nho SJ. Association between preoperative patient factors and clinically meaningful outcomes after hip arthroscopy for femoroacetabular impingement syndrome: a machine learning analysis. Am J Sports Med. 2022;50(3):746–56. This study is of importance as it is one of the few in sports medicine which attempts to predict outcomes beyond the MCID and also investigated prediction of the PASS and SCB.PubMedCrossRef
35.
Kunze KN, Polce EM, Clapp I, Nwachukwu BU, Chahla J, Nho SJ. Machine learning algorithms predict functional improvement after hip arthroscopy for femoroacetabular impingement syndrome in athletes. J Bone Joint Surg Am. 2021;103(12):1055–62.PubMedCrossRef
36.
Kunze KN, Polce EM, Ranawat AS, Randsborg PH, Williams RJ 3rd, Allen AA, et al. Application of machine learning algorithms to predict clinically meaningful improvement after arthroscopic anterior cruciate ligament reconstruction. Orthop J Sports Med. 2021;9(10):23259671211046576.PubMedPubMedCentral
37.
Ye Z, Zhang T, Wu C, Qiao Y, Su W, Chen J, et al. Predicting the objective and subjective clinical outcomes of anterior cruciate ligament reconstruction: a machine learning analysis of 432 patients. Am J Sports Med. 2022;50(14):3786–95.PubMedCrossRef
38.
Martin RK, Wastvedt S, Pareek A, Persson A, Visnes H, Fenstad AM, et al. Predicting subjective failure of ACL reconstruction: a machine learning analysis of the Norwegian Knee Ligament Register and patient reported outcomes. J ISAKOS. 2022;7(3):1–9.PubMedCrossRef
39.
Kumar V, Roche C, Overman S, Simovitch R, Flurin PH, Wright T, et al. Using machine learning to predict clinical outcomes after shoulder arthroplasty with a minimal feature set. J Shoulder Elbow Surg. 2021;30(5):e225–36.PubMedCrossRef
40.
Kumar V, Roche C, Overman S, Simovitch R, Flurin PH, Wright T, et al. What is the accuracy of three different machine learning techniques to predict clinical outcomes after shoulder arthroplasty? Clin Orthop Relat Res. 2020;478(10):2351–63.PubMedPubMedCentralCrossRef
41.
Ramkumar PN, Karnuta JM, Haeberle HS, Owusu-Akyaw KA, Warner TS, Rodeo SA, et al. Association between preoperative mental health and clinically meaningful outcomes after osteochondral allograft for cartilage defects of the knee: a machine learning analysis. Am J Sports Med. 2021;49(4):948–57.PubMedCrossRef
42.
Ramkumar PN, Karnuta JM, Haeberle HS, Rodeo SA, Nwachukwu BU, Williams RJ. Effect of preoperative imaging and patient factors on clinically meaningful outcomes and quality of life after osteochondral allograft transplantation: a machine learning analysis of cartilage defects of the knee. Am J Sports Med. 2021;49(8):2177–86.PubMedCrossRef
43.
Alaiti RK, Vallio CS, Assunção JH, Andrade e Silva FBd, Gracitelli MEC, Neto AAF, et al. Using machine learning to predict nonachievement of clinically significant outcomes after rotator cuff repair. Orthop J Sports Med. 2023;11(10):23259671231206180.
44.
Potty AG, Potty ASR, Maffulli N, Blumenschein LA, Ganta D, Mistovich RJ, et al. Approaching artificial intelligence in orthopaedics: predictive analytics and machine learning to prognosticate arthroscopic rotator cuff surgical outcomes. J Clin Med. 2023;12(6).
45.
••Kunze KN, Kaidi A, Madjarova S, Polce EM, Ranawat AS, Nawabi DH, et al. External validation of a machine learning algorithm for predicting clinically meaningful functional improvement after arthroscopic hip preservation surgery. Am J Sports Med. 2022;50(13):3593–9. This study is of great importance as it is the only external validation study in sports medicine to date with a primary outcome of clinically meaningful outcome achievement.PubMedCrossRef
46.
Merali ZG, Witiw CD, Badhiwala JH, Wilson JR, Fehlings MG. Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy. PLoS ONE. 2019;14(4): e0215133.PubMedPubMedCentralCrossRef
47.
Khan O, Badhiwala JH, Witiw CD, Wilson JR, Fehlings MG. Machine learning algorithms for prediction of health-related quality-of-life after surgery for mild degenerative cervical myelopathy. Spine J. 2021;21(10):1659–69.PubMedCrossRef
48.
Zhang JK, Jayasekera D, Javeed S, Greenberg JK, Blum J, Dibble CF, et al. Diffusion basis spectrum imaging predicts long-term clinical outcomes following surgery in cervical spondylotic myelopathy. Spine J. 2023;23(4):504–12.PubMedCrossRef
49.
Park C, Mummaneni PV, Gottfried ON, Shaffrey CI, Tang AJ, Bisson EF, et al. Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study. Neurosurg Focus. 2023;54(6):E5.PubMedCrossRef
50.
Staartjes VE, de Wispelaere MP, Vandertop WP, Schroder ML. Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar discectomy: feasibility of center-specific modeling. Spine J. 2019;19(5):853–61.PubMedCrossRef
51.
Pedersen CF, Andersen MO, Carreon LY, Eiskjaer S. Applied machine learning for spine surgeons: predicting outcome for patients undergoing treatment for lumbar disc herniation using PRO data. Global Spine J. 2022;12(5):866–76.PubMedCrossRef
52.
Berjano P, Langella F, Ventriglia L, Compagnone D, Barletta P, Huber D, et al. The influence of baseline clinical status and surgical strategy on early good to excellent result in spinal lumbar arthrodesis: a machine learning approach. J Pers Med. 2021;11(12).
53.
Staartjes VE, Stumpo V, Ricciardi L, Maldaner N, Eversdijk HAJ, Vieli M, et al. FUSE-ML: development and external validation of a clinical prediction model for mid-term outcomes after lumbar spinal fusion for degenerative disease. Eur Spine J. 2022;31(10):2629–38.PubMedCrossRef
54.
Karhade AV, Fogel HA, Cha TD, Hershman SH, Doorly TP, Kang JD, et al. Development of prediction models for clinically meaningful improvement in PROMIS scores after lumbar decompression. Spine J. 2021;21(3):397–404.PubMedCrossRef
55.
•Halicka M, Wilby M, Duarte R, Brown C. Predicting patient-reported outcomes following lumbar spine surgery: development and external validation of multivariable prediction models. BMC Musculoskelet Disord. 2023;24(1):333. This study is of importance given its attempt to externally validate their study findings in a unique population of patients.PubMedPubMedCentralCrossRef
56.
Siccoli A, de Wispelaere MP, Schroder ML, Staartjes VE. Machine learning-based preoperative predictive analytics for lumbar spinal stenosis. Neurosurg Focus. 2019;46(5):E5.PubMedCrossRef
57.
Brinkman N, Shah R, Doornberg J, Ring D, Gwilym S, Jayakumar P. Artificial neural networks outperform linear regression in estimating 9-month patient-reported outcomes after upper extremity fractures with increasing number of variables. OTA Int. 2023;6(5 Suppl): e284.PubMed
58.
Loos NL, Hoogendam L, Souer JS, Slijper HP, Andrinopoulou ER, Coppieters MW, et al. Machine learning can be used to predict function but not pain after surgery for thumb carpometacarpal osteoarthritis. Clin Orthop Relat Res. 2022;480(7):1271–84.PubMedPubMedCentralCrossRef
59.
Harrison CJ, Geoghegan L, Sidey-Gibbons CJ, Stirling PHC, McEachan JE, Rodrigues JN. Developing machine learning algorithms to support patient-centered, value-based carpal tunnel decompression surgery. Plast Reconstr Surg Glob Open. 2022;10(4): e4279.PubMedPubMedCentralCrossRef
60.
•Oeding JF, Krych AJ, Pearle AD, Kelly BT, Kunze KN. Medical imaging applications developed using artificial intelligence demonstrate high internal validity yet are limited in scope and lack external validation. Arthroscopy. 2024. This study is of importance to this topic as it parellels the themes identified in this review pertaining to clinically significant outcome achievement - repetitive use cases of current statistical methods and methodological shortcomings are currently impeding progress in this domain.
61.
Rossi MJ, Brand JC, Lubowitz JH. Minimally clinically important difference (MCID) is a low bar. Arthroscopy. 2023;39(2):139–41.PubMedCrossRef
62.
Kunze KN, Madjarova S, Jayakumar P, Nwachukwu BU. Challenges and opportunities for the use of patient-reported outcome measures in orthopaedic pediatric and sports medicine surgery. J Am Acad Orthop Surg. 2023;31(20):e898–905.PubMedCrossRef
Metadata
Title
Artificial Intelligence for Clinically Meaningful Outcome Prediction in Orthopedic Research: Current Applications and Limitations
Authors
Seong Jun Jang
Jake Rosenstadt
Eugenia Lee
Kyle N. Kunze
Publication date
08-04-2024
Publisher
Springer US
Published in
Current Reviews in Musculoskeletal Medicine / Issue 6/2024
Electronic ISSN: 1935-9748
DOI
https://doi.org/10.1007/s12178-024-09893-z