Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

Open Access 17-06-2023 | Original Article

Non-rigid point cloud registration for middle ear diagnostics with endoscopic optical coherence tomography

Authors: Peng Liu, Jonas Golde, Joseph Morgenstern, Sebastian Bodenstedt, Chenpan Li, Yujia Hu, Zhaoyu Chen, Edmund Koch, Marcus Neudert, Stefanie Speidel

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 1/2024

Abstract

Purpose

Middle ear infection is the most prevalent inflammatory disease, especially among the pediatric population. Current diagnostic methods are subjective and depend on visual cues from an otoscope, which is limited for otologists to identify pathology. To address this shortcoming, endoscopic optical coherence tomography (OCT) provides both morphological and functional in vivo measurements of the middle ear. However, due to the shadow of prior structures, interpretation of OCT images is challenging and time-consuming. To facilitate fast diagnosis and measurement, improvement in the readability of OCT data is achieved by merging morphological knowledge from ex vivo middle ear models with OCT volumetric data, so that OCT applications can be further promoted in daily clinical settings.

Methods

We propose C2P-Net: a two-staged non-rigid registration pipeline for complete to partial point clouds, which are sampled from ex vivo and in vivo OCT models, respectively. To overcome the lack of labeled training data, a fast and effective generation pipeline in Blender3D is designed to simulate middle ear shapes and extract in vivo noisy and partial point clouds.

Results

We evaluate the performance of C2P-Net through experiments on both synthetic and real OCT datasets. The results demonstrate that C2P-Net is generalized to unseen middle ear point clouds and capable of handling realistic noise and incompleteness in synthetic and real OCT data.

Conclusions

In this work, we aim to enable diagnosis of middle ear structures with the assistance of OCT images. We propose C2P-Net: a two-staged non-rigid registration pipeline for point clouds to support the interpretation of in vivo noisy and partial OCT images for the first time. Code is available at: https://gitlab.com/nct_tso_public/c2p-net.

Zwislocki J (1982) Normal function of the middle ear and its measurement. Audiology 21(1):4–14CrossRefPubMed

Won J, Monroy GL, Dsouza RI, Spillman DR, McJunkin J, Porter RG, Shi J, Aksamitiene E, Sherwood M, Stiger L, Boppart SA (2021) Handheld briefcase optical coherence tomography with real-time machine learning classifier for middle ear infections. Biosensors 11(5):143–156CrossRefPubMedPubMedCentral

Won J, Porter RG, Novak MA, Youakim J, Sum A, Barkalifa R, Aksamitiene E, Zhang A, Nolan R, Shelton R, Boppart SA (2021) In vivo dynamic characterization of the human tympanic membrane using pneumatic optical coherence tomography. J Biophotonics 14(4):1–12CrossRef

Snow JB, Wackym PA, Ballenger JJ (2009) Ballenger’s otorhinolaryngology: head and neck surgery. BC Decker, Hamilton, ON

Morgenstern J, Schindler M, Kirsten L, Golde J, Bornitz M, Kemper M, Koch E, Zahnert T, Neudert M (2020) Endoscopic optical coherence tomography for evaluation of success of tympanoplasty. Otol Neurotol 41(7):901–905CrossRef

Monroy GL, Won J, Dsouza R, Pande P, Hill MC, Porter RG, Novak MA, Spillman DR, Boppart SA (2019) Automated classification platform for the identification of otitis media using optical coherence tomography. NPJ Dig Med 2(1):1–22

Monroy GL, Shelton RL, Nolan RM, Nguyen CT, Novak MA, Hill MC, McCormick DT, Boppart SA (2015) Noninvasive depth-resolved optical measurements of the tympanic membrane and middle ear for differentiating otitis media. Laryngoscope 125(8):276–282CrossRef

Brock A, Lim T, Ritchie JM, Weston NJ (2016) Generative and discriminative voxel modeling with convolutional neural networks. In: Neural inofrmation processing conference: 3D deep learning

Su H, Maji S, Kalogerakis E, Learned-Miller E (2015) Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE international conference on computer vision, p 945–953

10.

Feng Y, Zhang Z, Zhao X, Ji R, Gao Y (2018) Gvcnn: group-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 264–272

11.

Qi CR, Su H, Nießner M, Dai A, Yan M, Guibas LJ (2016) Volumetric and multi-view cnns for object classification on 3d data. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 5648–5656

12.

Qi CR, Su H, Mo K, Guibas LJ (2017) Pointnet: deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, p 652–660

13.

Qi CR, Yi L, Su H, Guibas LJ (2017) Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in neural information processing systems, vol 30

14.

Thomas H, Qi CR, Deschaud J-E, Marcotegui B, Goulette F, Guibas LJ (2019) Kpconv: flexible and deformable convolution for point clouds. In: Proceedings of the IEEE/CVF international conference on computer vision, p 6411–6420

15.

Xu Y, Fan T, Xu M, Zeng L, Qiao Y (2018) Spidercnn: deep learning on point sets with parameterized convolutional filters. In: Proceedings of the European conference on computer vision (ECCV), p 87–102

16.

Choy C, Park J, Koltun V (2019) Fully convolutional geometric features. In: Proceedings of the IEEE/CVF international conference on computer vision, p 8958–8966

17.

Besl PJ, McKay ND (1992) Method for registration of 3-d shapes. Sensor fusion IV: control paradigms and data structures, vol 1611. Spie, Bellingham, pp 586–606

18.

Amberg B, Romdhani S, Vetter T (2007) Optimal step nonrigid icp algorithms for surface registration. In: 2007 IEEE conference on computer vision and pattern recognition, IEEE, p 1–8

19.

Aoki Y, Goforth H, Srivatsan RA, Lucey S (2019) Pointnetlk: robust & efficient point cloud registration using pointnet. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p 7163–7172

20.

Yew ZJ, Lee GH (2022) Regtr: end-to-end point cloud correspondences with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p 6677–6686

21.

Zhu L, Guan H, Lin C, Han R (2022) Neighborhood-aware geometric encoding network for point cloud registration. arXiv preprint arXiv:2201.12094

22.

Li Y, Harada T (2022) Non-rigid point cloud registration with neural deformation pyramid. arXiv preprint arXiv:2205.12796

23.

Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: IJCAI’81: 7th international joint conference on artificial intelligence, p 674–679

Title: Non-rigid point cloud registration for middle ear diagnostics with endoscopic optical coherence tomography
Authors: Peng Liu
Jonas Golde
Joseph Morgenstern
Sebastian Bodenstedt
Chenpan Li
Yujia Hu
Zhaoyu Chen
Edmund Koch
Marcus Neudert
Stefanie Speidel
Publication date: 17-06-2023
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 1/2024
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-023-02960-9

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Non-rigid point cloud registration for middle ear diagnostics with endoscopic optical coherence tomography

Abstract

Purpose

Methods

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2024

A cGAN-based network for depth estimation from bronchoscopic images

Development and assessment of novel assist device for cardiac cannulation

Development of an anatomical breast phantom from polyvinyl chloride plastisol with lesions of various shape, elasticity and echogenicity for teaching ultrasound examination

Ultrasound guidance in navigated liver surgery: toward deep-learning enhanced compensation of deformation and organ motion

Deep learning-based age estimation from chest CT scans

Validation of real-time inside-out tracking and depth realization technologies for augmented reality-based neuronavigation