Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-01-2019 | Original Article

Deep learning-based detection of motion artifacts in probe-based confocal laser endomicroscopy images

Authors: Marc Aubreville, Maike Stoeve, Nicolai Oetter, Miguel Goncalves, Christian Knipfer, Helmut Neumann, Christopher Bohr, Florian Stelzle, Andreas Maier

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 1/2019

Abstract

Purpose:

Probe-based confocal laser endomicroscopy (pCLE) is a subcellular in vivo imaging technique capable of producing images that enable diagnosis of malign structural modifications in epithelial tissue. Images acquired with pCLE are, however, often tainted by significant artifacts that impair diagnosis. This is especially detrimental for automated image analysis, which is why said images are often excluded from recognition pipelines.

Methods

We present an approach for the automatic detection of motion artifacts in pCLE images and apply this methodology to a data set of 15 thousand images of epithelial tissue acquired in the oral cavity and the vocal folds. The approach is based on transfer learning from intermediate endpoints within a pre-trained Inception v3 network with tailored preprocessing. For detection within the non-rectangular pCLE images, we perform pooling within the activation maps of the network and evaluate this at different network depths.

Results

We achieved area under the ROC curve values of 0.92 with the proposed method, compared to 0.80 for the best feature-based machine learning approach. Our overall accuracy with the presented approach is 94.8%.

Conclusion

Over traditional machine learning approaches with state-of-the-art features, we achieved significantly improved overall performance.

https://www5.cs.fau.de/~aubreville/.

Abbaci M, Breuskin I, Casiraghi O, De Leeuw F (2014) Confocal laser endomicroscopy for non-invasive head and neck cancer imaging: a comprehensive review. Oral Oncol 50(8):711–6. https://doi.org/10.1016/j.oraloncology.2014.05.002 CrossRefPubMed

Agaimy A, Weichert W (2016) Grading of head and neck neoplasms. Der Pathol 37(4):285–292. https://doi.org/10.1007/s00292-016-0173-9 CrossRef

Araujo H, Dias JM (1996) An introduction to the log-polar mapping (image sampling). In: Proceedings 2nd workshop on cybernetic vision. IEEE, pp 139–144. https://doi.org/10.1109/CYBVIS.1996.629454

Aubreville M, Knipfer C, Oetter N, Jaremenko Christian, Rodner E, Denzler J, Bohr C, Neumann H, Stelzle F, Maier A (2017) Automatic classification of cancerous tissue in laserendomicroscopy images of the oral cavity using deep learning. Sci Rep 7(1):11979. https://doi.org/10.1038/s41598-017-12320-8 CrossRefPubMedPubMedCentral

Bier B, Mualla F, Steidl S, Bohr C, Neumann H, Maier A, Hornegger J (2015) Band-pass filter design by segmentation in frequency domain for detection of epithelial cells in endomicroscope images. In: Bildverarbeitung für die Medizin. Springer, Berlin, pp 413–418. https://doi.org/10.1007/978-3-662-46224-9_71

Chauhan SS, Dayyeh BKA, Bhat YM, Gottlieb KT, Hwang JH, Komanduri S, Konda V, Lo SK, Manfredi MA, Maple JT, Murad FM, Siddiqui UD, Banerjee S, Wallace MB (2014) Confocal laser endomicroscopy. Gastrointest Endosc 80(6):928–938. https://doi.org/10.1016/j.gie.2014.06.021 CrossRef

Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2013 16(Pt 2):411–418. https://doi.org/10.1007/978-3-642-40763-5_51

Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, CVPR 2005. 1:886–893. https://doi.org/10.1109/CVPR.2005.177

Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848

10.

Deniz O, García GB, Salido J, De la Torre F (2011) Face recognition using histograms of oriented gradients. Pattern Recognit Lett 32(12):1598–1603. https://doi.org/10.1016/j.patrec.2011.01.004 CrossRef

11.

Dittberner A, Rodner E, Ortmann W, Stadler J, Schmidt C, Petersen I, Stallmach A, Denzler J, Guntinas-Lichius O (2016) Automated analysis of confocal laser endomicroscopy images to detect head and neck cancer. Head Neck 38(S1):E1419–E1426. https://doi.org/10.1002/hed.24253 CrossRefPubMed

12.

Gil D, Ramos-Terrades O, Minchole E, Sanchez C, de Frutos NC, Diez-Ferrer M, Ortiz RM, Rosell A (2017) Classification of confocal endomicroscopy patterns for diagnosis of lung cancer. In: Medical image computing and computer-assisted intervention—MICCAI 2017, pp. 151–159. Springer, Cham. https://doi.org/10.1007/978-3-319-67543-5_15

13.

Goncalves M, Iro H, Dittberner A, Agaimy A, Bohr C (2017) Value of confocal laser endomicroscopy in the diagnosis of vocal cord lesions. Eur Rev Med Pharmacol Sci 21:3990–3997PubMed

14.

Hong J, Park By, Park H (2017) Convolutional neural network classifier for distinguishing Barrett’s esophagus and neoplasia endomicroscopy images. In: 39th Annual International conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 2892–2895. https://doi.org/10.1109/EMBC.2017.8037461

15.

Izadyyazdanabadi M, Belykh E, Cavallo C, Zhao X, Gandhi S, Moreira LB, Eschbacher J, Nakaji P, Preul MC, Yang Y (2018) Weakly-supervised learning-based feature localization in confocal laser endomicroscopy glioma images. arXiv preprint arXiv:1804.09428v1

16.

Izadyyazdanabadi M, Belykh E, Martirosyan N, Eschbacher J, Nakaji P, Yang Y, Preul MC (2017)Improving utility of brain tumor confocal laser endomicroscopy - objective value assessment and diagnostic frame detection with convolutional neural networks. In: Medical Imaging 2017, vol 10134. SPIE. https://doi.org/10.1117/12.2254902

17.

Izadyyazdanabadi M, Belykh E, Mooney M, Eschbacher J, Nakaji P, Yang Y, Preul MC (2018) Prospects for theranostics in neurosurgical technology—empowering confocal laser endomicroscopy diagnostics via deep learning. arXiv preprint arXiv:1804.09873

18.

Izadyyazdanabadi M, Belykh E, Mooney M, Martirosyan N, Eschbacher J, Nakaji P, Preul MC, Yang Y (2018) Convolutional neural networks: ensemble modeling, fine-tuning and unsupervised semantic localization for neurosurgical cle images. J Vis Commun Image Represent 54:10–20. https://doi.org/10.1016/j.jvcir.2018.04.004 CrossRef

19.

Jaremenko C, Maier A, Steidl S, Hornegger J, Oetter N, Knipfer C, Stelzle F, Neumann H (2015) Classification of confocal laser endomicroscopic images of the oral cavity to distinguish pathological from healthy tissue. In: Bildverarbeitung für die Medizin 2015. Springer, Berlin, pp 479–485. https://doi.org/10.1007/978-3-662-46224-9_82

20.

Kingma D, Ba J (2014) ADAM: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

21.

Laemmel E, Genet M, Le Goualher G, Perchant A, Le Gargasson JF, Vicaut E (2004) Fibered confocal fluorescence microscopy (Cell-viZio) facilitates extended imaging in the field of microcirculation. A comparison with intravital microscopy. J Vasc Res 41(5):400–411. https://doi.org/10.1159/000081209 CrossRefPubMed

22.

Macé V, Ahluwalia A, Coron E, Le Rhun M, Boureille A, Bossard C, Mosnier JF, Matysiak-Budnik T, Tarnawski AS (2015) Confocal laser endomicroscopy: a new gold standard for the assessment of mucosal healing in ulcerative colitis. J Gastroenterol Hepatol 30:85–92. https://doi.org/10.1111/jgh.12748 CrossRefPubMed

23.

Maier AK, Schebesch F, Syben C, Würfl T, Steidl S, Choi JH, Fahrig R (2017) Precision learning: towards use of known operators in neural networks. arXiv preprint arXiv:1712.00374

24.

Maier H, Dietz A, Gewelke U, Heller W, Weidauer H (1992) Tobacco and alcohol and the risk of head and neck cancer. Clin Investig 70(3–4):320–327. https://doi.org/10.1007/BF00184668 CrossRefPubMed

25.

Muto M, Nakane M, Katada C, Sano Y, Ohtsu A, Esumi H, Ebihara S, Yoshida S (2004) Squamous cell carcinoma in situ at oropharyngeal and hypopharyngeal mucosal sites. Cancer 101(6):1375–1381. https://doi.org/10.1002/cncr.20482 CrossRefPubMed

26.

Nachalon Y, Alkan U, Shvero J, Yaniv D, Shkedy Y, Limon D, Popovtzer A (2017) Assessment of laryngeal cancer in patients younger than 40 years. The Laryngoscope. https://doi.org/10.1002/lary.26951 CrossRefPubMed

27.

Neumann H, Langner C, Neurath MF, Vieth M (2012) Confocal laser endomicroscopy for diagnosis of Barrett’s Esophagus. Front Oncol 2:42. https://doi.org/10.3389/fonc.2012.00042 CrossRefPubMedPubMedCentral

28.

Neumann H, Vieth M, Atreya R, Neurath MF, Mudter J (2011) Prospective evaluation of the learning curve of confocal laser endomicroscopy in patients with IBD. Histol Histopathol 26(7):867–872PubMed

29.

Oetter N, Knipfer C, Rohde M, Wilmowsky C, Maier A, Brunner K, Adler W, Neukam FW, Neumann H, Stelzle F (2016) Development and validation of a classification and scoring system for the diagnosis of oral squamous cell carcinomas through confocal laser endomicroscopy. J Transl Med 14(1):1–11. https://doi.org/10.1186/s12967-016-0919-4 CrossRef

30.

Oquab M, Bottou L, Laptev I, Sivic J (2015) Is object localization for free? Weakly-supervised learning with convolutional neural networks. In: Proceedings of IEEE conference on computer vision and pattern recognition. IEEE, pp 685–694. https://doi.org/10.1109/CVPR.2015.7298668

31.

Parikh ND, Gibson J, Nagar A, Ahmed AA, Aslanian HR (2016) Confocal laser endomicroscopy features of sessile serrated adenomas/polyps. U Eur Gastroenterol J 4(4):599–603. https://doi.org/10.1177/2050640615621819 CrossRef

32.

Pavlov V, Meyronet D, Meyer-Bisch V, Armoiry X, Pikul B, Dumot C, Beuriat P-A, Signorelli F, Guyotat J (2016) Intraoperative probe-based confocal laser endomicroscopy in surgery and stereotactic biopsy of low-grade and high-grade gliomas. Neurosurgery 79(4):604–612. https://doi.org/10.1227/NEU.0000000000001365 CrossRefPubMed

33.

Robert Koch Institut. Zentrum für Krebsregisterdaten (2017) Krebs in Deutschland für 2013/2014, 11th edn. Robert Koch Institut, Berlin

34.

Stoeve M, Aubreville M, Oetter N, Knipfer C, Neumann H, Stelzle F, Maier A (2018) Motion Artifact Detection in Confocal Laser Endomicroscopy Images. In: Bildverarbeitung für die Medizin. Springer, Berlin, pp 328–333. https://doi.org/10.1007/978-3-662-56537-7_85

35.

Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: The IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2015.7298594

36.

Vo K, Jaremenko Christian, Bohr C, Neumann H, Maier A (2017) Automatic classification and pathological staging of confocal laser endomicroscopic images of the vocal cords. In: Bildverarbeitung für die Medizin 2017. Springer, Berlin, pp 312–317. https://doi.org/10.1007/978-3-662-54345-0_70

37.

Westra WH (2015) The pathology of HPV-related head and neck cancer: implications for the diagnostic pathologist. Semin Diagn Pathol 32(1):42–53. https://doi.org/10.1053/j.semdp.2015.02.023 CrossRefPubMed

38.

Wiesner C, Jäger W, Salzer A, Biesterfeld S, Kiesslich R, Hampel C, Thüroff JW, Goetz M (2010) Confocal laser endomicroscopy for the diagnosis of urothelial bladder neoplasia: a technology of the future? BJU Int 107(3):399–403. https://doi.org/10.1111/j.1464-410X.2010.09540.x CrossRefPubMed

39.

Xing F, Xie Y, Su H, Liu F, Yang L (2017) Deep learning in microscopy image analysis: a survey. IEEE Trans Neural Netw Learn Syst PP(99):1–19. https://doi.org/10.1109/TNNLS.2017.2766168

40.

Yang Y, Li T, Li W, Wu H, Fan W, Zhang W (2017) Lesion detection and grading of diabetic retinopathy via two-stages deep convolutional neural networks. In: Medical image computing and computer-assisted intervention—MICCAI 2017. Springer, Cham, pp 533–540. https://doi.org/10.1007/978-3-319-66179-7_61

Title: Deep learning-based detection of motion artifacts in probe-based confocal laser endomicroscopy images
Authors: Marc Aubreville
Maike Stoeve
Nicolai Oetter
Miguel Goncalves
Christian Knipfer
Helmut Neumann
Christopher Bohr
Florian Stelzle
Andreas Maier
Publication date: 01-01-2019
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 1/2019
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-018-1836-1

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Deep learning-based detection of motion artifacts in probe-based confocal laser endomicroscopy images

Abstract

Purpose:

Methods

Results

Conclusion

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Purpose:

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2019

Automatic bone segmentation in whole-body CT images

Volumetric analysis of intracranial vessels: a novel tool for evaluation of cerebral vasospasm

MR-conditional steerable needle robot for intracerebral hemorrhage removal

Standard navigation versus intraoperative computed tomography navigation in upper cervical spine trauma

Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients

The accuracy of image-based safety analysis for robotic cochlear implantation