Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
European Radiology
Published in: European Radiology 10/2022

27-06-2022 | Artificial Intelligence | Imaging Informatics and Artificial Intelligence

Diagnostic accuracy and potential covariates of artificial intelligence for diagnosing orthopedic fractures: a systematic literature review and meta-analysis

Authors: Xiang Zhang, Yi Yang, Yi-Wei Shen, Ke-Rui Zhang, Ze-kun Jiang, Li-Tai Ma, Chen Ding, Bei-Yu Wang, Yang Meng, Hao Liu

Published in: European Radiology | Issue 10/2022

Login to get access

Abstract

Objectives

To systematically quantify the diagnostic accuracy and identify potential covariates affecting the performance of artificial intelligence (AI) in diagnosing orthopedic fractures.

Methods

PubMed, Embase, Web of Science, and Cochrane Library were systematically searched for studies on AI applications in diagnosing orthopedic fractures from inception to September 29, 2021. Pooled sensitivity and specificity and the area under the receiver operating characteristic curves (AUC) were obtained. This study was registered in the PROSPERO database prior to initiation (CRD 42021254618).

Results

Thirty-nine were eligible for quantitative analysis. The overall pooled AUC, sensitivity, and specificity were 0.96 (95% CI 0.94–0.98), 90% (95% CI 87–92%), and 92% (95% CI 90–94%), respectively. In subgroup analyses, multicenter designed studies yielded higher sensitivity (92% vs. 88%) and specificity (94% vs. 91%) than single-center studies. AI demonstrated higher sensitivity with transfer learning (with vs. without: 92% vs. 87%) or data augmentation (with vs. without: 92% vs. 87%), compared to those without. Utilizing plain X-rays as input images for AI achieved results comparable to CT (AUC 0.96 vs. 0.96). Moreover, AI achieved comparable results to humans (AUC 0.97 vs. 0.97) and better results than non-expert human readers (AUC 0.98 vs. 0.96; sensitivity 95% vs. 88%).

Conclusions

AI demonstrated high accuracy in diagnosing orthopedic fractures from medical images. Larger-scale studies with higher design quality are needed to validate our findings.

Key Points

• Multicenter study design, application of transfer learning, and data augmentation are closely related to improving the performance of artificial intelligence models in diagnosing orthopedic fractures.
• Utilizing plain X-rays as input images for AI to diagnose fractures achieved results comparable to CT (AUC 0.96 vs. 0.96).
• AI achieved comparable results to humans (AUC 0.97 vs. 0.97) but was superior to non-expert human readers (AUC 0.98 vs. 0.96, sensitivity 95% vs. 88%) in diagnosing fractures.
Appendix
Available only for authorised users
Literature
1.
2.
3.
Sahlin Y (1990) Occurrence of fractures in a defined population: a 1-year study. Injury 21:158–160PubMedCrossRef
4.
Çolak I, Bekler HI, Bulut G, Eceviz E, Gülabi D, Çeçen GS (2018) Lack of experience is a significant factor in the missed diagnosis of perilunate fracture dislocation or isolated dislocation. Acta Orthop Traumatol Turc 52:32–36
5.
Moonen PJ, Mercelina L, Boer W, T Fret (2017) Diagnostic error in the Emergency Department: follow up of patients with minor trauma in the outpatient clinic. Scand J Trauma Resusc Emerg Med 25:13
6.
Wei CJ, Tsai WC, Tiu CM, Wu HT, Chiou HJ, Chang CY (2006) Systematic analysis of missed extremity fractures in emergency radiology. Acta Radiol 47:710–717
7.
Bottle A, Aylin P (2006) Mortality associated with delay in operation after hip fracture: observational study. BMJ 332:947–951
8.
Leer-Salvesen S, Engesæter LB, Dybvik E, Furnes O, Kristensen TB, Gjertsen JE (2019) Does time from fracture to surgery affect mortality and intraoperative medical complications for hip fracture patients? An observational study of 73 557 patients reported to the Norwegian Hip Fracture Register. Bone Joint J 101-b:1129-1137
9.
McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94PubMedCrossRef
10.
Rodríguez-Ruiz A, Krupinski E, Mordang JJ et al (2019) Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 290:305–314PubMedCrossRef
11.
Rodriguez-Ruiz A, Lång K, Gubern-Merida A et al (2019) Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. J Natl Cancer Inst 111:916–922PubMedPubMedCentralCrossRef
12.
Schmidt-Erfurth U, Sadeghipour A, Gerendas BS, Waldstein SM, Bogunović H (2018) Artificial intelligence in retina. Prog Retin Eye Res 67:1–29
13.
Vujosevic S, Aldington SJ, Silva P et al (2020) Screening for diabetic retinopathy: new perspectives and challenges. Lancet Diabetes Endocrinol 8:337–347PubMedCrossRef
14.
Kikinis R, Wells WM 3rd (2020) Detection of brain metastases with deep learning single-shot detector algorithms. Radiology 295:416–417PubMedCrossRef
15.
Xue J, Wang B, Ming Y et al (2020) Deep learning-based detection and segmentation-assisted management of brain metastases. Neuro Oncol 22:505–514PubMedCrossRef
16.
Abbasi J (2020) Artificial intelligence-based skin cancer phone apps unreliable. JAMA 323:1336PubMed
17.
18.
Gregory J, Welliver S, Chong J (2020) Top 10 reviewer critiques of radiology artificial intelligence (AI) articles: qualitative thematic analysis of reviewer critiques of machine learning/deep learning manuscripts submitted to JMRI. J Magn Reson Imaging 52:248–254PubMedCrossRef
19.
Park SH, Han K (2018) Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology 286:800–809PubMedCrossRef
20.
Park SH, Kressel HY (2018) Connecting technological innovation in artificial intelligence to real-world medical practice through rigorous clinical validation: what peer-reviewed medical journals could do. J Korean Med Sci 33:e152PubMedPubMedCentralCrossRef
21.
Duron L, Ducarouge A, Gillibert A et al (2021) Assessment of an AI aid in detection of adult appendicular skeletal fractures by emergency physicians and radiologists: a multicenter cross-sectional diagnostic study. Radiology 300:120–129PubMedCrossRef
22.
Kirienko M, Sollini M, Ninatti G et al (2021) Distributed learning: a reliable privacy-preserving strategy to change multicenter collaborations using AI. Eur J Nucl Med Mol Imaging 48:3791–3804PubMedPubMedCentralCrossRef
23.
Lee AY, Yanagihara RT, Lee CS et al (2021) Multicenter, head-to-head, real-world validation study of seven automated artificial intelligence diabetic retinopathy screening systems. Diabetes Care 44:1168–1175PubMedPubMedCentralCrossRef
24.
Novakovsky G, Saraswat M, Fornes O, Mostafavi S, Wasserman WW (2021) Biologically relevant transfer learning improves transcription factor binding prediction. Genome Biol 22:280
25.
Shi H, Li J, Mao, Hwang KS (2021) Lateral transfer learning for multiagent reinforcement learning. IEEE Trans Cybern 1–13
26.
Xiao Y, Liang F, Liu B (2022) A transfer learning-based multi-instance learning method with weak labels. IEEE Trans Cybern 52:287–300PubMedCrossRef
27.
Zhen L, Hu P, Peng X, Goh RSM, Zhou JT (2022) Deep multimodal transfer learning for cross-modal retrieval. IEEE Trans Neural Netw Learn Syst 33:798–810
28.
Chaitanya K, Karani N, Baumgartner CF et al (2021) Semi-supervised task-driven data augmentation for medical image segmentation. Med Image Anal 68:101934PubMedCrossRef
29.
Gao J, Hua Y, Hu G, Wang C, Robertson NM (2021) Discrepancy-guided domain-adaptive data augmentation. IEEE Trans Neural Netw Learn Syst 1–12
30.
Tran NT, Tran VH, Nguyen NB, Nguyen TK, Cheung NM (2021) On data augmentation for GAN training. IEEE Trans Image Process 30:1882–1897
31.
Jonsdottir KY, Østergaard L, Mouridsen K (2009) Predicting tissue outcome from acute stroke magnetic resonance imaging: improving model performance by optimal sampling of training data. Stroke 40:3006–3011PubMedCrossRef
32.
Rank N, Pfahringer B, Kempfert J et al (2020) Deep-learning-based real-time prediction of acute kidney injury outperforms human predictive performance. NPJ Digit Med 3:139PubMedPubMedCentralCrossRef
33.
Sanders WS, Johnston CI, Bridges SM, Burgess SC, Willeford KO (2011) Prediction of cell penetrating peptides by support vector machines. PLoS Comput Biol 7:e1002101
34.
Hashimoto DA, Witkowski E, Gao L, Meireles O, Rosman G (2020) Artificial intelligence in anesthesiology: current techniques, clinical applications, and limitations. Anesthesiology 132:379–394
35.
Kumar A, Pirogova E, Mahmoud SS, Fang Q (2021) Classification of error-related potentials evoked during stroke rehabilitation training. J Neural Eng 18
36.
Reichstein M, Camps-Valls G, Stevens B et al (2019) Deep learning and process understanding for data-driven Earth system science. Nature 566:195–204PubMedCrossRef
37.
Schwendicke F, Golla T, Dreher M, Krois J (2019) Convolutional neural networks for dental image diagnostics: A scoping review. J Dent 91:103226
38.
39.
40.
Liberati A, Altman DG, Tetzlaff J et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6:e1000100PubMedPubMedCentralCrossRef
41.
Whiting PF, Rutjes AW, Westwood ME et al (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155:529–536PubMedCrossRef
42.
Higgins JP, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. BMJ 327:557–560
43.
Bae J, Yu S, Oh J et al (2021) External validation of deep learning algorithm for detecting and visualizing femoral neck fracture including displaced and non-displaced fracture on plain X-ray. J Digit Imaging 34:1099–1109PubMedCrossRef
44.
Beyaz S, Açıcı K, Sümer E (2020) Femoral neck fracture detection in X-ray images using deep learning and genetic algorithm approaches. Jt Dis Relat Surg 31:175–183PubMedPubMedCentral
45.
Blüthgen C, Becker AS, Vittoria DMI, Meier A, Martini K, Frauenfelder T (2020) Detection and localization of distal radius fractures: deep learning system versus radiologists. Eur J Radiol 126:108925
46.
Cheng CT, Chen CC, Cheng FJ et al (2020) A human-algorithm integration system for hip fracture detection on plain radiography: system development and validation study. JMIR Med Inform 8:e19416PubMedPubMedCentralCrossRef
47.
Cheng CT, Ho TY, Lee TY et al (2019) Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs. Eur Radiol 29:5469–5477PubMedPubMedCentralCrossRef
48.
Choi J, Hui JZ, Spain D, Su YS, Cheng CT, Liao CH(2021) Practical computer vision application to detect hip fractures on pelvic X-rays: a bi-institutional study. Trauma Surg Acute Care Open 6:e000705
49.
Choi JW, Cho YJ, Lee S et al (2020) Using a dual-input convolutional neural network for automated detection of pediatric supracondylar fracture on conventional radiography. Invest Radiol 55:101–110PubMedCrossRef
50.
Chung SW, Han SS, Lee JW et al (2018) Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 89:468–473PubMedPubMedCentralCrossRef
51.
Derkatch S, Kirby C, Kimelman D, Jozani MJ, Davidson JM, Leslie WD (2019) Identification of vertebral fractures by convolutional neural networks to predict nonvertebral and hip fractures: a registry-based cohort study of dual X-ray absorptiometry. Radiology 293:405–411
52.
Gan K, Xu D, Lin Y et al (2019) Artificial intelligence detection of distal radius fractures: a comparison between the convolutional neural network and professional assessments. Acta Orthop 90:394–400PubMedPubMedCentralCrossRef
53.
Guy S, Jacquet C, Tsenkoff D, Argenson JN, Ollivier M (2021) Deep learning for the radiographic diagnosis of proximal femur fractures: limitations and programming issues. Orthop Traumatol Surg Res 107:102837
54.
Hendrix N, Scholten E, Vernhout B et al (2021) Development and validation of a convolutional neural network for automated detection of scaphoid fractures on conventional radiographs. Radiol Artif Intell 3:e200260PubMedPubMedCentralCrossRef
55.
Jiménez-Sánchez A, Kazi A, Albarqouni S et al (2020) Precise proximal femur fracture classification for interactive training and surgical planning. Int J Comput Assist Radiol Surg 15:847–857PubMedCrossRef
56.
Jones RM, Sharma A, Hotchkiss R et al (2020) Assessment of a deep-learning system for fracture detection in musculoskeletal radiographs. NPJ Digit Med 3:144PubMedPubMedCentralCrossRef
57.
Kim MW, Jung J, Park SJ et al (2021) Application of convolutional neural networks for distal radio-ulnar fracture detection on plain radiographs in the emergency room. Clin Exp Emerg Med 8:120–127PubMedPubMedCentralCrossRef
58.
Kitamura G, Chung CY, Moore BEN (2019) Ankle fracture detection utilizing a convolutional neural network ensemble implemented with a small sample, de novo training, and multiview incorporation. J Digit Imaging 32:672–677PubMedPubMedCentralCrossRef
59.
Krogue JD, Cheng KV, Hwang KM et al (2020) Automatic hip fracture identification and functional subclassification with deep learning. Radiol Artif Intell 2:e190023PubMedPubMedCentralCrossRef
60.
Langerhuizen DWG, Bulstra AEJ, Janssen SJ et al (2020) Is deep learning on par with human observers for detection of radiographically visible and occult fractures of the scaphoid? Clin Orthop Relat Res 478:2653–2659PubMedPubMedCentralCrossRef
61.
Li YC, Chen HH, Horng-Shing LH, Wu HTH, Chang MC, Chou PH (2021) Can a deep-learning model for the automated detection of vertebral fractures approach the performance level of human subspecialists? Clin Orthop Relat Res 479:1598–1612
62.
Ma Y, Luo Y (2021) Bone fracture detection through the two-stage system of Crack-Sensitive Convolutional Neural Network. Inform Med Unlocked 22
63.
MacKinnon T (2018) Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol 73:439–445PubMedCrossRef
64.
Mawatari T, Hayashida Y, Katsuragawa S et al (2020) The effect of deep convolutional neural networks on radiologists' performance in the detection of hip fractures on digital pelvic radiographs. Eur J Radiol 130:109188PubMedCrossRef
65.
Mehta SD, Sebro R (2020) Computer-aided detection of incidental lumbar spine fractures from routine dual-energy X-ray absorptiometry (DEXA) studies using a support vector machine (SVM) classifier. J Digit Imaging 33:204–210PubMedCrossRef
66.
Monchka BA, Kimelman D, Lix LM, Leslie WD (2021) Feasibility of a generalized convolutional neural network for automated identification of vertebral compression fractures: the Manitoba Bone Mineral Density Registry. Bone 150:116017
67.
Mutasa S, Varada S, Goel A, Wong TT, Rasiej MJ (2020) Advanced deep learning techniques applied to automated femoral neck fracture detection and classification. J Digit Imaging 33:1209–1217
68.
Ozkaya E, Topal FE, Bulut T, Gursoy M, Ozuysal M, Karakaya Z (2022) Evaluation of an artificial intelligence system for diagnosing scaphoid fracture on direct radiography. Eur J Trauma Emerg Surg 48:585–592
69.
Rayan JC, Reddy N, Kan JH, Zhang W, Annapragada A (2019) Binomial classification of pediatric elbow fractures using a deep learning multiview approach emulating radiologist decision making. Radiol Artif Intell 1:e180015
70.
Reichert G, Bellamine A, Fontaine M et al (2021) How can a deep learning algorithm improve fracture detection on X-rays in the emergency room? J Imaging 7
71.
Ren M, Yi PH (2022) Deep learning detection of subtle fractures using staged algorithms to mimic radiologist search pattern. Skeletal Radiol 51:345–353PubMedCrossRef
72.
Sato Y, Takegami Y, Asamoto T et al (2021) Artificial intelligence improves the accuracy of residents in the diagnosis of hip fractures: a multicenter study. BMC Musculoskelet Disord 22:407PubMedPubMedCentralCrossRef
73.
Urakawa T, Tanaka Y, Goto S, Matsuzawa H, Watanabe K, Endo N (2019) Detecting intertrochanteric hip fractures with orthopedist-level accuracy using a deep convolutional neural network. Skeletal Radiol 48:239–244
74.
Yoon AP, Lee YL, Kane RL, Kuo CF, Lin C, Chung KC (2021) Development and validation of a deep learning model using convolutional neural networks to identify scaphoid fractures in radiographs. JAMA Netw Open 4:e216096
75.
Yu JS, Yu SM, Erdal BS et al (2020) Detection and localisation of hip fractures on anteroposterior radiographs with artificial intelligence: proof of concept. Clin Radiol 75:237.e231-237.e239
76.
Al-Helo S, Alomari RS, Ghosh S et al (2013) Compression fracture diagnosis in lumbar: a clinical CAD system. Int J Comput Assist Radiol Surg 8:461–469PubMedCrossRef
77.
Burns JE, Yao J, Summers RM (2017) Vertebral body compression fractures and bone density: automated detection and classification on CT images. Radiology 284:788–797PubMedCrossRef
78.
Hu Y, He X, Zhang R, Guo L, Gao L, Wang J (2021) Slice grouping and aggregation network for auxiliary diagnosis of rib fractures. Biomed Signal Process Control 67
79.
Small JE, Osler P, Paul AB, Kunst M (2021) CT cervical spine fracture detection using a convolutional neural network. AJNR Am J Neuroradiol 42:1341–1347
80.
Voter AF, Larson ME, Garrett JW, Yu JPJ (2021) Diagnostic accuracy and failure mode analysis of a deep learning algorithm for the detection of cervical spine fractures. AJNR Am J Neuroradiol 42:1550–1556
81.
Weikert T, Noordtzij LA, Bremerich J et al (2020) Assessment of a deep learning algorithm for the detection of rib fractures on whole-body trauma computed tomography. Korean J Radiol 21:891–899PubMedCrossRef
82.
Caravagna G, Giarratano Y, Ramazzotti D et al (2018) Detecting repeated cancer evolution from multi-region tumor sequencing data. Nat Methods 15:707–714PubMedPubMedCentralCrossRef
83.
84.
85.
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359CrossRef
86.
Thrall JH, Li X, Li Q et al (2018) Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success. J Am Coll Radiol 15:504–508PubMedCrossRef
87.
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization 2017 IEEE International Conference on Computer Vision (ICCV), pp 618-626
88.
89.
Metadata
Title
Diagnostic accuracy and potential covariates of artificial intelligence for diagnosing orthopedic fractures: a systematic literature review and meta-analysis
Authors
Xiang Zhang
Yi Yang
Yi-Wei Shen
Ke-Rui Zhang
Ze-kun Jiang
Li-Tai Ma
Chen Ding
Bei-Yu Wang
Yang Meng
Hao Liu
Publication date
27-06-2022
Publisher
Springer Berlin Heidelberg
Published in
European Radiology / Issue 10/2022
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-022-08956-4

Other articles of this Issue 10/2022

European Radiology 10/2022 Go to the issue

Hepatobiliary-Pancreas

Correlation of transcriptional subtypes with a validated CT radiomics score in resectable pancreatic ductal adenocarcinoma

Magnetic Resonance

Machine learning for the automatic assessment of aortic rotational flow and wall shear stress from 4D flow cardiac magnetic resonance imaging

Interventional

Optimising CT-guided biopsies of sclerotic bone lesions in cancer patients

Gastrointestinal

Contrast-enhanced ultrasound of solid pancreatic head lesions: a prospective study

Correction

Correction to: Rotational abnormalities in dysplastic hips and how to predict acetabular torsion

Gastrointestinal

Preoperative prediction of inadvertent enterotomy during adhesive small bowel obstruction surgery using combination of CT features