Top

Archives of Orthopaedic and Trauma Surgery

Published in:

Open Access 05-04-2024 | Distal Radius Fracture | Handsurgery

Diagnostic power of ChatGPT 4 in distal radius fracture detection through wrist radiographs

Authors: Sinan Mert, Patrick Stoerzer, Johannes Brauer, Benedikt Fuchs, Elisabeth M. Haas-Lützenberger, Wolfram Demmer, Riccardo E. Giunta, Tim Nuernberger

Published in: Archives of Orthopaedic and Trauma Surgery | Issue 5/2024

Abstract

Distal radius fractures rank among the most prevalent fractures in humans, necessitating accurate radiological imaging and interpretation for optimal diagnosis and treatment. In addition to human radiologists, artificial intelligence systems are increasingly employed for radiological assessments. Since 2023, ChatGPT 4 has offered image analysis capabilities, which can also be used for the analysis of wrist radiographs. This study evaluates the diagnostic power of ChatGPT 4 in identifying distal radius fractures, comparing it with a board-certified radiologist, a hand surgery resident, a medical student, and the well-established AI Gleamer BoneView™. Results demonstrate ChatGPT 4’s good diagnostic accuracy (sensitivity 0.88, specificity 0.98, diagnostic power (AUC) 0.93), surpassing the medical student (sensitivity 0.98, specificity 0.72, diagnostic power (AUC) 0.85; p = 0.04) significantly. Nevertheless, the diagnostic power of ChatGPT 4 lags behind the hand surgery resident (sensitivity 0.99, specificity 0.98, diagnostic power (AUC) 0.985; p = 0.014) and Gleamer BoneView™(sensitivity 1.00, specificity 0.98, diagnostic power (AUC) 0.99; p = 0.006). This study highlights the utility and potential applications of artificial intelligence in modern medicine, emphasizing ChatGPT 4 as a valuable tool for enhancing diagnostic capabilities in the field of medical imaging.

Available only for authorised users

Rupp M, Walter N, Pfeifer C, Lang S, Kerschbaum M, Krutsch W, Baumann F, Alt V (2021) The incidence of Fractures among the Adult Population of Germany-an analysis from 2009 through 2019. Dtsch Arztebl Int 118(40):665–669. https://doi.org/10.3238/arztebl.m2021.0238CrossRefPubMedPubMedCentral

Amin S, Achenbach SJ, Atkinson EJ, Khosla S, Melton LJ 3rd (2014) Trends in fracture incidence: a population-based study over 20 years. J Bone Min Res 29(3):581–589. https://doi.org/10.1002/jbmr.2072CrossRef

Lichtman DM, Bindra RR, Boyer MI et al (2010) Treatment of distal radius fractures. J Am Acad Orthop Surg 18(3):180–189. https://doi.org/10.5435/00124635-201003000-00007CrossRefPubMed

Cavalcanti Kussmaul A, Kuehlein T, Langer MF, Ayache A, Unglaub F (2023) The treatment of closed finger and metacarpal fractures. Dtsch Arztebl Int 120(50):855–862. https://doi.org/10.3238/arztebl.m2023.0226CrossRefPubMedPubMedCentral

de Putter CE, Selles RW, Polinder S, Panneman MJM, Hovius SER, van Beeck EF (2012) Economic impact of hand and wrist injuries: health-care costs and productivity costs in a population-based study. J Bone Joint Surg Am 94(9):e56. https://doi.org/10.2106/JBJS.K.00561CrossRefPubMed

Waever D, Madsen ML, Rölfing JHD, Borris LC, Henriksen M, Nagel LL, Thorninger R (2018) Distal radius fractures are difficult to classify. Injury 49 Suppl 1S29–S32. https://doi.org/10.1016/S0020-1383(18)30299-7

Guermazi A, Tannoury C, Kompel AJ et al (2022) Improving Radiographic fracture Recognition Performance and Efficiency using Artificial Intelligence. Radiology 302(3):627–636. https://doi.org/10.1148/radiol.210937CrossRefPubMed

Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, Chartash D (2023) How does ChatGPT perform on the United States Medical Licensing examination? The implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med Educ 9:e45312. https://doi.org/10.2196/45312CrossRefPubMedPubMedCentral

Russe MF, Fink A, Ngo H, Tran H, Bamberg F, Reisert M, Rau A (2023) Performance of ChatGPT, human radiologists, and context-aware ChatGPT in identifying AO codes from radiology reports. Sci Rep 13(1):14215. https://doi.org/10.1038/s41598-023-41512-8CrossRefPubMedPubMedCentral

10.

OpenAI J, Achiam SA, Sandhini Agarwal et al (2023) GPT-4 Technical Report. arXiv:2303–08774 [cs.CL]. https://doi.org/10.48550/arXiv.2303.08774

11.

Long Ouyang JW, Xu Jiang D, Almeida et al (2022) Training language models to follow instructions with human feedback. https://doi.org/10.48550/arXiv.2203.02155. arXiv:2203.02155 [cs.CL]

12.

Lyu Q, Tan J, Zapadka ME, Ponnatapura J, Niu C, Myers KJ, Wang G, Whitlow CT (2023) Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential. Vis Comput Ind Biomed Art 6(1):9. https://doi.org/10.1186/s42492-023-00136-5CrossRefPubMedPubMedCentral

13.

Jeblick K, Schachtner B, Dexl J et al (2023) ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports. Eur Radiol. https://doi.org/10.1007/s00330-023-10213-1CrossRefPubMed

14.

Li H, Moon JT, Iyer D et al (2023) Decoding radiology reports: potential application of OpenAI ChatGPT to enhance patient understanding of diagnostic reports. Clin Imaging 101:137–141. https://doi.org/10.1016/j.clinimag.2023.06.008CrossRefPubMed

15.

Barash Y, Klang E, Konen E, Sorin V (2023) ChatGPT-4 assistance in optimizing Emergency Department Radiology referrals and Imaging Selection. J Am Coll Radiol 20(10):998–1003. https://doi.org/10.1016/j.jacr.2023.06.009CrossRefPubMed

16.

Rao A, Kim J, Kamineni M, Pang M, Lie W, Succi MD (2023) Evaluating ChatGPT as an Adjunct for Radiologic decision-making. medRxiv. https://doi.org/10.1101/2023.02.02.23285399CrossRefPubMedPubMedCentral

17.

Huang Y, Gomaa A, Semrau S et al (2023) Benchmarking ChatGPT-4 on a radiation oncology in-training exam and Red Journal Gray Zone cases: potentials and challenges for Ai-assisted medical education and decision making in radiation oncology. Front Oncol 13:1265024. https://doi.org/10.3389/fonc.2023.1265024CrossRefPubMedPubMedCentral

18.

Patil NS, Huang RS, van der Pol CB, Larocque N (2023) Using Artificial Intelligence Chatbots as a radiologic decision-making Tool for Liver Imaging: do ChatGPT and Bard communicate information consistent with the ACR appropriateness Criteria? J Am Coll Radiol 20(10):1010–1013. https://doi.org/10.1016/j.jacr.2023.07.010CrossRefPubMed

19.

Oka K, Shiode R, Yoshii Y, Tanaka H, Iwahashi T, Murase T (2021) Artificial intelligence to diagnosis distal radius fracture using biplane plain X-rays. J Orthop Surg Res 16(1):694. https://doi.org/10.1186/s13018-021-02845-0CrossRefPubMedPubMedCentral

20.

Tobler P, Cyriac J, Kovacs BK et al (2021) AI-based detection and classification of distal radius fractures using low-effort data labeling: evaluation of applicability and effect of training set size. Eur Radiol 31(9):6816–6824. https://doi.org/10.1007/s00330-021-07811-2CrossRefPubMedPubMedCentral

21.

Zech JR, Carotenuto G, Igbinoba Z, Tran CV, Insley E, Baccarella A, Wong TT (2023) Detecting pediatric wrist fractures using deep-learning-based object detection. Pediatr Radiol 53(6):1125–1134. https://doi.org/10.1007/s00247-023-05588-8CrossRefPubMed

22.

Olczak J, Pavlopoulos J, Prijs J, Ijpma FFA, Doornberg JN, Lundström C, Hedlund J, Gordon M (2021) Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal. Acta Orthop 92(5):513–525. https://doi.org/10.1080/17453674.2021.1918389CrossRefPubMedPubMedCentral

23.

Anttila TT, Karjalainen TV, Mäkelä TO, Waris EM, Lindfors NC, Leminen MM, Ryhänen JO (2023) Detecting Distal Radius fractures using a segmentation-based Deep Learning Model. J Digit Imaging 36(2):679–687. https://doi.org/10.1007/s10278-022-00741-5CrossRefPubMed

24.

Gan K, Xu D, Lin Y et al (2019) Artificial intelligence detection of distal radius fractures: a comparison between the convolutional neural network and professional assessments. Acta Orthop 90(4):394–400. https://doi.org/10.1080/17453674.2019.1600125CrossRefPubMedPubMedCentral

25.

Kim DH, MacKinnon T (2018) Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol 73(5):439–445. https://doi.org/10.1016/j.crad.2017.11.015CrossRefPubMed

26.

Thian YL, Li Y, Jagmohan P, Sia D, Chan VEY, Tan RT (2019) Convolutional Neural Networks for Automated Fracture Detection and localization on wrist radiographs. Radiol Artif Intell 1(1):e180001. https://doi.org/10.1148/ryai.2019180001CrossRefPubMedPubMedCentral

27.

Lindsey R, Daluiski A, Chopra S et al (2018) Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci U S A 115(45):11591–11596. https://doi.org/10.1073/pnas.1806905115CrossRefPubMedPubMedCentral

28.

Castillo C, Steffens T, Sim L, Caffery L (2021) The effect of clinical information on radiology reporting: a systematic review. J Med Radiat Sci 68(1):60–74. https://doi.org/10.1002/jmrs.424CrossRefPubMed

29.

Raisuddin AM, Vaattovaara E, Nevalainen M et al (2021) Critical evaluation of deep neural networks for wrist fracture detection. Sci Rep 11(1):6006. https://doi.org/10.1038/s41598-021-85570-2CrossRefPubMedPubMedCentral

30.

Sander AL, Leiblein M, Sommer K, Marzi I, Schneidmuller D, Frank J (2020) Epidemiology and treatment of distal radius fractures: current concept based on fracture severity and not on age. Eur J Trauma Emerg Surg 46(3):585–590. https://doi.org/10.1007/s00068-018-1023-7CrossRefPubMed

31.

Koo OT, Tan DM, Chong AK (2013) Distal radius fractures: an epidemiological review. Orthop Surg 5(3):209–213. https://doi.org/10.1111/os.12045CrossRefPubMedPubMedCentral

Title: Diagnostic power of ChatGPT 4 in distal radius fracture detection through wrist radiographs
Authors: Sinan Mert
Patrick Stoerzer
Johannes Brauer
Benedikt Fuchs
Elisabeth M. Haas-Lützenberger
Wolfram Demmer
Riccardo E. Giunta
Tim Nuernberger
Publication date: 05-04-2024
Publisher: Springer Berlin Heidelberg
Keywords: Distal Radius Fracture
Artificial Intelligence
Published in: Archives of Orthopaedic and Trauma Surgery / Issue 5/2024
Print ISSN: 0936-8051
Electronic ISSN: 1434-3916
DOI: https://doi.org/10.1007/s00402-024-05298-2

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Diagnostic power of ChatGPT 4 in distal radius fracture detection through wrist radiographs

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 5/2024

Eighteen-year outcome of anterior cruciate ligament reconstruction with patellar tendon or hamstring autograft

Comparison of bilateral cementless total knee arthroplasty results between patients in different BMI groups

Long-term recurrence of Dupuytren’s disease treated with clostridium histolitycum collagenase. Surgical treatment and anatomopathological study

Patient reported outcome measures (PROMs) as primary and secondary outcomes in total hip and knee arthroplasty randomized controlled trials: a systematic review

Cat bite injuries to the hand and forearm: the impact of antibiotic treatment on microbiological findings and clinical outcome

Mid-term results of an anatomic total knee replacement design