Skip to main content
Top
Published in: Skeletal Radiology 8/2020

Open Access 01-08-2020 | Artificial Intelligence | Scientific Article

Deep convolutional neural network-based detection of meniscus tears: comparison with radiologists and surgery as standard of reference

Authors: Benjamin Fritz, Giuseppe Marbach, Francesco Civardi, Sandro F. Fucentese, Christian W.A. Pfirrmann

Published in: Skeletal Radiology | Issue 8/2020

Login to get access

Abstract

Objective

To clinically validate a fully automated deep convolutional neural network (DCNN) for detection of surgically proven meniscus tears.

Materials and methods

One hundred consecutive patients were retrospectively included, who underwent knee MRI and knee arthroscopy in our institution. All MRI were evaluated for medial and lateral meniscus tears by two musculoskeletal radiologists independently and by DCNN. Included patients were not part of the training set of the DCNN. Surgical reports served as the standard of reference. Statistics included sensitivity, specificity, accuracy, ROC curve analysis, and kappa statistics.

Results

Fifty-seven percent (57/100) of patients had a tear of the medial and 24% (24/100) of the lateral meniscus, including 12% (12/100) with a tear of both menisci. For medial meniscus tear detection, sensitivity, specificity, and accuracy were for reader 1: 93%, 91%, and 92%, for reader 2: 96%, 86%, and 92%, and for the DCNN: 84%, 88%, and 86%. For lateral meniscus tear detection, sensitivity, specificity, and accuracy were for reader 1: 71%, 95%, and 89%, for reader 2: 67%, 99%, and 91%, and for the DCNN: 58%, 92%, and 84%. Sensitivity for medial meniscus tears was significantly different between reader 2 and the DCNN (p = 0.039), and no significant differences existed for all other comparisons (all p ≥ 0.092). The AUC-ROC of the DCNN was 0.882, 0.781, and 0.961 for detection of medial, lateral, and overall meniscus tear. Inter-reader agreement was very good for the medial (kappa = 0.876) and good for the lateral meniscus (kappa = 0.741).

Conclusion

DCNN-based meniscus tear detection can be performed in a fully automated manner with a similar specificity but a lower sensitivity in comparison with musculoskeletal radiologists.
Literature
1.
go back to reference Baker BE, Peckham AC, Pupparo F, Sanborn JC. Review of meniscal injury and associated sports. Am J Sports Med. 1985;13(1):1–4.CrossRef Baker BE, Peckham AC, Pupparo F, Sanborn JC. Review of meniscal injury and associated sports. Am J Sports Med. 1985;13(1):1–4.CrossRef
2.
go back to reference Majewski M, Susanne H, Klaus S. Epidemiology of athletic knee injuries: a 10-year study. Knee. 2006;13(3):184–8.CrossRef Majewski M, Susanne H, Klaus S. Epidemiology of athletic knee injuries: a 10-year study. Knee. 2006;13(3):184–8.CrossRef
3.
go back to reference Kumm J, Roemer FW, Guermazi A, Turkiewicz A, Englund M. Natural history of intrameniscal signal intensity on knee MR images: six years of data from the osteoarthritis initiative. Radiology. 2016;278(1):164–71.CrossRef Kumm J, Roemer FW, Guermazi A, Turkiewicz A, Englund M. Natural history of intrameniscal signal intensity on knee MR images: six years of data from the osteoarthritis initiative. Radiology. 2016;278(1):164–71.CrossRef
4.
go back to reference MacFarlane LA, Yang H, Collins JE, Guermazi A, Jones MH, Teeple E, et al. Associations among meniscal damage, meniscal symptoms and knee pain severity. Osteoarthr Cartil. 2017;25(6):850–7.CrossRef MacFarlane LA, Yang H, Collins JE, Guermazi A, Jones MH, Teeple E, et al. Associations among meniscal damage, meniscal symptoms and knee pain severity. Osteoarthr Cartil. 2017;25(6):850–7.CrossRef
5.
go back to reference Hede A, Larsen E, Sandberg H. The long term outcome of open total and partial meniscectomy related to the quantity and site of the meniscus removed. Int Orthop. 1992;16(2):122–5.CrossRef Hede A, Larsen E, Sandberg H. The long term outcome of open total and partial meniscectomy related to the quantity and site of the meniscus removed. Int Orthop. 1992;16(2):122–5.CrossRef
6.
go back to reference Howell JR, Handoll HH. Surgical treatment for meniscal injuries of the knee in adults. Cochrane Database Syst Rev. 2000;2:CD001353. Howell JR, Handoll HH. Surgical treatment for meniscal injuries of the knee in adults. Cochrane Database Syst Rev. 2000;2:CD001353.
7.
go back to reference Beaufils P, Pujol N. Management of traumatic meniscal tear and degenerative meniscal lesions. Save the meniscus. Orthop Traumatol Surg Res. 2017;103(8S):S237–44.CrossRef Beaufils P, Pujol N. Management of traumatic meniscal tear and degenerative meniscal lesions. Save the meniscus. Orthop Traumatol Surg Res. 2017;103(8S):S237–44.CrossRef
8.
go back to reference McCarty EC, Marx RG, Wickiewicz TL. Meniscal tears in the athlete. Operative and nonoperative management. Phys Med Rehabil Clin N Am. 2000;11(4):867–80.CrossRef McCarty EC, Marx RG, Wickiewicz TL. Meniscal tears in the athlete. Operative and nonoperative management. Phys Med Rehabil Clin N Am. 2000;11(4):867–80.CrossRef
9.
go back to reference DeHaven KE, Black KP, Griffiths HJ. Open meniscus repair. Technique and two to nine year results. Am J Sports Med. 1989;17(6):788–95.CrossRef DeHaven KE, Black KP, Griffiths HJ. Open meniscus repair. Technique and two to nine year results. Am J Sports Med. 1989;17(6):788–95.CrossRef
10.
go back to reference Brophy RH, Matava MJ. Surgical options for meniscal replacement. J Am Acad Orthop Surg. 2012;20(5):265–72.CrossRef Brophy RH, Matava MJ. Surgical options for meniscal replacement. J Am Acad Orthop Surg. 2012;20(5):265–72.CrossRef
11.
go back to reference Stein T, Mehling AP, Welsch F, von Eisenhart-Rothe R, Jager A. Long-term outcome after arthroscopic meniscal repair versus arthroscopic partial meniscectomy for traumatic meniscal tears. Am J Sports Med. 2010;38(8):1542–8.CrossRef Stein T, Mehling AP, Welsch F, von Eisenhart-Rothe R, Jager A. Long-term outcome after arthroscopic meniscal repair versus arthroscopic partial meniscectomy for traumatic meniscal tears. Am J Sports Med. 2010;38(8):1542–8.CrossRef
12.
go back to reference Bonamo JJ, Kessler KJ, Noah J. Arthroscopic meniscectomy in patients over the age of 40. Am J Sports Med. 1992;20(4):422–8 discussion 428-429.CrossRef Bonamo JJ, Kessler KJ, Noah J. Arthroscopic meniscectomy in patients over the age of 40. Am J Sports Med. 1992;20(4):422–8 discussion 428-429.CrossRef
13.
go back to reference Khan M, Evaniew N, Bedi A, Ayeni OR, Bhandari M. Arthroscopic surgery for degenerative tears of the meniscus: a systematic review and meta-analysis. CMAJ. 2014;186(14):1057–64.CrossRef Khan M, Evaniew N, Bedi A, Ayeni OR, Bhandari M. Arthroscopic surgery for degenerative tears of the meniscus: a systematic review and meta-analysis. CMAJ. 2014;186(14):1057–64.CrossRef
14.
go back to reference Zikria B, Hafezi-Nejad N, Roemer FW, Guermazi A, Demehri S. Meniscal surgery: risk of radiographic joint space narrowing progression and subsequent knee replacement-data from the osteoarthritis initiative. Radiology. 2017;282(3):807–16.CrossRef Zikria B, Hafezi-Nejad N, Roemer FW, Guermazi A, Demehri S. Meniscal surgery: risk of radiographic joint space narrowing progression and subsequent knee replacement-data from the osteoarthritis initiative. Radiology. 2017;282(3):807–16.CrossRef
15.
go back to reference Naraghi AM, White LM. Imaging of athletic injuries of knee ligaments and menisci: sports imaging series. Radiology. 2016;281(1):23–40.CrossRef Naraghi AM, White LM. Imaging of athletic injuries of knee ligaments and menisci: sports imaging series. Radiology. 2016;281(1):23–40.CrossRef
16.
go back to reference Crawford R, Walley G, Bridgman S, Maffulli N. Magnetic resonance imaging versus arthroscopy in the diagnosis of knee pathology, concentrating on meniscal lesions and ACL tears: a systematic review. Br Med Bull. 2007;84:5–23.CrossRef Crawford R, Walley G, Bridgman S, Maffulli N. Magnetic resonance imaging versus arthroscopy in the diagnosis of knee pathology, concentrating on meniscal lesions and ACL tears: a systematic review. Br Med Bull. 2007;84:5–23.CrossRef
17.
go back to reference Oei EH, Nikken JJ, Verstijnen AC, Ginai AZ, Myriam Hunink MG. MR imaging of the menisci and cruciate ligaments: a systematic review. Radiology. 2003;226(3):837–48.CrossRef Oei EH, Nikken JJ, Verstijnen AC, Ginai AZ, Myriam Hunink MG. MR imaging of the menisci and cruciate ligaments: a systematic review. Radiology. 2003;226(3):837–48.CrossRef
18.
go back to reference Mazurowski MA, Buda M, Saha A, Bashir MR. Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI. J Magn Reson Imaging 2018. Mazurowski MA, Buda M, Saha A, Bashir MR. Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI. J Magn Reson Imaging 2018.
19.
go back to reference Pesapane F, Codari M, Sardanelli F. Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine. Eur Radiol Exp. 2018;2(1):35.CrossRef Pesapane F, Codari M, Sardanelli F. Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine. Eur Radiol Exp. 2018;2(1):35.CrossRef
20.
go back to reference Wang S, Summers RM. Machine learning and radiology. Med Image Anal. 2012;16(5):933–51.CrossRef Wang S, Summers RM. Machine learning and radiology. Med Image Anal. 2012;16(5):933–51.CrossRef
21.
go back to reference Weikert T, Cyriac J, Yang S, Nesic I, Parmar V, Stieltjes B. A practical guide to artificial intelligence-based image analysis in radiology. Invest Radiol. 2019. Weikert T, Cyriac J, Yang S, Nesic I, Parmar V, Stieltjes B. A practical guide to artificial intelligence-based image analysis in radiology. Invest Radiol. 2019.
22.
go back to reference Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition, 2014. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition, 2014.
23.
go back to reference Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1. Lake Tahoe, Nevada: Curran Associates Inc. 2012:1097–1105. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1. Lake Tahoe, Nevada: Curran Associates Inc. 2012:1097–1105.
24.
go back to reference He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV): IEEE Computer Society 2015:1026–1034. He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV): IEEE Computer Society 2015:1026–1034.
25.
go back to reference Fritz J, Fritz B, Thawait GG, Meyer H, Gilson WD, Raithel E. Three-dimensional CAIPIRINHA SPACE TSE for 5-minute high-resolution MRI of the knee. Investig Radiol. 2016;51(10):609–17.CrossRef Fritz J, Fritz B, Thawait GG, Meyer H, Gilson WD, Raithel E. Three-dimensional CAIPIRINHA SPACE TSE for 5-minute high-resolution MRI of the knee. Investig Radiol. 2016;51(10):609–17.CrossRef
26.
go back to reference Fritz J, Fritz B, Zhang J, Thawait GK, Joshi DH, Pan L, et al. Simultaneous multislice accelerated turbo spin echo magnetic resonance imaging: comparison and combination with in-plane parallel imaging acceleration for high-resolution magnetic resonance imaging of the knee. Investig Radiol. 2017;52(9):529–37.CrossRef Fritz J, Fritz B, Zhang J, Thawait GK, Joshi DH, Pan L, et al. Simultaneous multislice accelerated turbo spin echo magnetic resonance imaging: comparison and combination with in-plane parallel imaging acceleration for high-resolution magnetic resonance imaging of the knee. Investig Radiol. 2017;52(9):529–37.CrossRef
27.
go back to reference Fritz J, Raithel E, Thawait GK, Gilson W, Papp DF. Six-fold acceleration of high-spatial resolution 3D SPACE MRI of the knee through incoherent k-space undersampling and iterative reconstruction-first experience. Investig Radiol. 2016;51(6):400–9.CrossRef Fritz J, Raithel E, Thawait GK, Gilson W, Papp DF. Six-fold acceleration of high-spatial resolution 3D SPACE MRI of the knee through incoherent k-space undersampling and iterative reconstruction-first experience. Investig Radiol. 2016;51(6):400–9.CrossRef
28.
go back to reference Szegedy C, Wei L, Yangqing J, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2015 7–12 June 2015; 2015. p. 1–9. Szegedy C, Wei L, Yangqing J, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2015 7–12 June 2015; 2015. p. 1–9.
29.
go back to reference Richardson ML. The Zombie plot: a simple graphic method for visualizing the efficacy of a diagnostic test. AJR Am J Roentgenol. 2016;207(4):W43–52.CrossRef Richardson ML. The Zombie plot: a simple graphic method for visualizing the efficacy of a diagnostic test. AJR Am J Roentgenol. 2016;207(4):W43–52.CrossRef
30.
go back to reference Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.CrossRef Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.CrossRef
31.
go back to reference Lee H, Tajmir S, Lee J, Zissen M, Yeshiwas BA, Alkasab TK, et al. Fully automated deep learning system for bone age assessment. J Digit Imaging. 2017;30(4):427–41.CrossRef Lee H, Tajmir S, Lee J, Zissen M, Yeshiwas BA, Alkasab TK, et al. Fully automated deep learning system for bone age assessment. J Digit Imaging. 2017;30(4):427–41.CrossRef
32.
go back to reference Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, et al. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop. 2017;88(6):581–6.CrossRef Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, et al. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop. 2017;88(6):581–6.CrossRef
33.
go back to reference Couteaux V, Si-Mohamed S, Nempont O, Lefevre T, Popoff A, Pizaine G, et al. Automatic knee meniscus tear detection and orientation classification with Mask-RCNN. Diagn Interv Imaging. 2019;100(4):235–42.CrossRef Couteaux V, Si-Mohamed S, Nempont O, Lefevre T, Popoff A, Pizaine G, et al. Automatic knee meniscus tear detection and orientation classification with Mask-RCNN. Diagn Interv Imaging. 2019;100(4):235–42.CrossRef
34.
go back to reference Roblot V, Giret Y, Bou Antoun M, Morillot C, Chassin X, Cotten A, et al. Artificial intelligence to diagnose meniscus tears on MRI. Diagn Interv Imaging. 2019;100(4):243–9.CrossRef Roblot V, Giret Y, Bou Antoun M, Morillot C, Chassin X, Cotten A, et al. Artificial intelligence to diagnose meniscus tears on MRI. Diagn Interv Imaging. 2019;100(4):243–9.CrossRef
35.
go back to reference Bien N, Rajpurkar P, Ball RL, Irvin J, Park A, Jones E, et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med. 2018;15(11):e1002699.CrossRef Bien N, Rajpurkar P, Ball RL, Irvin J, Park A, Jones E, et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med. 2018;15(11):e1002699.CrossRef
36.
go back to reference Liu F, Zhou Z, Samsonov A, Blankenbaker D, Larison W, Kanarek A, et al. Deep learning approach for evaluating knee MR images: achieving high diagnostic performance for cartilage lesion detection. Radiology. 2018;289(1):160–9.CrossRef Liu F, Zhou Z, Samsonov A, Blankenbaker D, Larison W, Kanarek A, et al. Deep learning approach for evaluating knee MR images: achieving high diagnostic performance for cartilage lesion detection. Radiology. 2018;289(1):160–9.CrossRef
37.
go back to reference Liu F, Zhou Z, Jang H, Samsonov A, Zhao G, Kijowski R. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magn Reson Med. 2018;79(4):2379–91.CrossRef Liu F, Zhou Z, Jang H, Samsonov A, Zhao G, Kijowski R. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magn Reson Med. 2018;79(4):2379–91.CrossRef
38.
go back to reference Zhou Z, Zhao G, Kijowski R, Liu F. Deep convolutional neural network for segmentation of knee joint anatomy. Magn Reson Med. 2018;80(6):2759–70.CrossRef Zhou Z, Zhao G, Kijowski R, Liu F. Deep convolutional neural network for segmentation of knee joint anatomy. Magn Reson Med. 2018;80(6):2759–70.CrossRef
39.
go back to reference Tack A, Mukhopadhyay A, Zachow S. Knee menisci segmentation using convolutional neural networks: data from the osteoarthritis initiative. Osteoarthr Cartil. 2018;26(5):680–8.CrossRef Tack A, Mukhopadhyay A, Zachow S. Knee menisci segmentation using convolutional neural networks: data from the osteoarthritis initiative. Osteoarthr Cartil. 2018;26(5):680–8.CrossRef
40.
go back to reference Phelan N, Rowland P, Galvin R, O’Byrne JM. A systematic review and meta-analysis of the diagnostic accuracy of MRI for suspected ACL and meniscal tears of the knee. Knee Surg Sports Traumatol Arthrosc. 2016;24(5):1525–39.CrossRef Phelan N, Rowland P, Galvin R, O’Byrne JM. A systematic review and meta-analysis of the diagnostic accuracy of MRI for suspected ACL and meniscal tears of the knee. Knee Surg Sports Traumatol Arthrosc. 2016;24(5):1525–39.CrossRef
41.
go back to reference Richardson ML, Petscavage JM. An interactive web-based tool for detecting verification (work-up) bias in studies of the efficacy of diagnostic imaging. Acad Radiol. 2010;17(12):1580–3.CrossRef Richardson ML, Petscavage JM. An interactive web-based tool for detecting verification (work-up) bias in studies of the efficacy of diagnostic imaging. Acad Radiol. 2010;17(12):1580–3.CrossRef
Metadata
Title
Deep convolutional neural network-based detection of meniscus tears: comparison with radiologists and surgery as standard of reference
Authors
Benjamin Fritz
Giuseppe Marbach
Francesco Civardi
Sandro F. Fucentese
Christian W.A. Pfirrmann
Publication date
01-08-2020
Publisher
Springer Berlin Heidelberg
Published in
Skeletal Radiology / Issue 8/2020
Print ISSN: 0364-2348
Electronic ISSN: 1432-2161
DOI
https://doi.org/10.1007/s00256-020-03410-2

Other articles of this Issue 8/2020

Skeletal Radiology 8/2020 Go to the issue

Browser's Notes

Browser's notes