Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

16-05-2024 | Sinus Surgery | Original Article

An endoscopic chisel: intraoperative imaging carves 3D anatomical models

Authors: Jan Emily Mangulabnan, Roger D. Soberanis-Mukul, Timo Teufel, Manish Sahu, Jose L. Porras, S. Swaroop Vedula, Masaru Ishii, Gregory Hager, Russell H. Taylor, Mathias Unberath

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 7/2024

Abstract

Purpose

Preoperative imaging plays a pivotal role in sinus surgery where CTs offer patient-specific insights of complex anatomy, enabling real-time intraoperative navigation to complement endoscopy imaging. However, surgery elicits anatomical changes not represented in the preoperative model, generating an inaccurate basis for navigation during surgery progression.

Methods:

We propose a first vision-based approach to update the preoperative 3D anatomical model leveraging intraoperative endoscopic video for navigated sinus surgery where relative camera poses are known. We rely on comparisons of intraoperative monocular depth estimates and preoperative depth renders to identify modified regions. The new depths are integrated in these regions through volumetric fusion in a truncated signed distance function representation to generate an intraoperative 3D model that reflects tissue manipulation

Results:

We quantitatively evaluate our approach by sequentially updating models for a five-step surgical progression in an ex vivo specimen. We compute the error between correspondences from the updated model and ground-truth intraoperative CT in the region of anatomical modification. The resulting models show a decrease in error during surgical progression as opposed to increasing when no update is employed.

Conclusion:

Our findings suggest that preoperative 3D anatomical models can be updated using intraoperative endoscopy video in navigated sinus surgery. Future work will investigate improvements to monocular depth estimation as well as removing the need for external navigation systems. The resulting ability to continuously update the patient model may provide surgeons with a more precise understanding of the current anatomical state and paves the way toward a digital twin paradigm for sinus surgery

Available only for authorised users

Clark DW, Del Signore AG, Raithatha R, Senior BA (2018) Nasal airway obstruction: prevalence and anatomic contributors. Ear, Nose & Throat Journal 97(6):173–176CrossRef

Moubayed SP, Most SP (2022) Evaluation and management of the nasal airway. Clin Plast Surg 49(1):23–31CrossRefPubMed

Lee K-I, In SM, Kim J-Y, Hong J-Y, Han K-D, Kim J-S, Jung YG (2021) Association of nasal septal deviation with the incidence of anxiety, depression, and migraine: A national population-based study. PLoS ONE 16(11):0259468CrossRef

Wyler B, Mallon W (2019) Sinusitis update. Emerg Med Clin North Am 37:41–54. https://doi.org/10.1016/j.emc.2018.09.007CrossRefPubMed

Lethbridge-Cejku M, Schiller J, Bernadel L (2004) Summary health statistics for u.s. adults: national health interview survey, 2002. Vital and health statistics. Series 10. Data from the National Health Survey 10:1–151

Liu X, Sinha A, Ishii M, Hager GD, Reiter A, Taylor RH, Unberath M (2019) Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Trans Med Imaging 39(5):1438–1447CrossRefPubMedPubMedCentral

Liu, X, Stiber M, Huang J, Ishii M, Hager GD, Taylor RH, Unberath M (2020) Reconstructing sinus anatomy from endoscopic video–towards a radiation-free approach for quantitative longitudinal assessment. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020, pp. 3–13 . Springer

Liu X, Li Z, Ishii M, Hager GD, Taylor RH, Unberath M (2022) Sage: slam with appearance and geometry prior for endoscopy. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 5587–5593 . IEEE

Hernández I, Soberanis-Mukul R, Mangulabnan JE, Sahu M, Winter J, Vedula S, Ishii M, Hager G, Taylor RH, Unberath M (2023) Investigating keypoint descriptors for camera relocalization in endoscopy surgery. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-023-02918-xCrossRefPubMedPubMedCentral

10.

Chen F, Cui X, Han B, Liu J, Zhang X, Liao H (2021) Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy. Comput Methods Programs Biomed 201:105952CrossRefPubMed

11.

Hu J, Jin X, Lee JB, Zhang L, Chaudhary V, Guthikonda M, Yang KH, King AI (2007) Intraoperative brain shift prediction using a 3d inhomogeneous patient-specific finite element model. J Neurosurg 106(1):164–169CrossRefPubMed

12.

Miga MI, Paulsen KD, Hoopes PJ, Kennedy FE, Hartov A, Roberts DW (2000) In vivo quantification of a homogeneous brain deformation model for updating preoperative images during surgery. IEEE Trans Biomed Eng 47(2):266–273CrossRefPubMed

13.

François T, Calvet L, Sève-d’ Erceville C, Bourdel N, Bartoli A (2021) Image-based incision detection for topological intraoperative 3d model update in augmented reality assisted laparoscopic surgery. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, pp. 647–656 . Springer

14.

Chalasani P, Wang L, Roy R, Simaan N, Taylor RH, Kobilarov M (2016) Concurrent nonparametric estimation of organ geometry and tissue stiffness using continuous adaptive palpation. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 4164–4171 . IEEE

15.

Wang L, Chen Z, Chalasani P, Yasin RM, Kazanzides P, Taylor RH, Simaan N (2017) Force-controlled exploration for updating virtual fixture geometry in model-mediated telemanipulation. J Mech Robot 9(2):021010

16.

Yasin R, Chalasani P, Zevallos N, Shahbazi M, Li Z, Deguet A, Kazanzides P, Choset H, Taylor RH, Simaan N (2020) Evaluation of hybrid control and palpation assistance for situational awareness in telemanipulated task execution. IEEE Transactions on Medical Robotics and Bionics 3(1):31–43CrossRef

17.

Shi Y, Deng X, Tong Y, Li R, Zhang Y, Ren L, Si W (2022) Synergistic digital twin and holographic augmented-reality-guided percutaneous puncture of respiratory liver tumor. IEEE Trans on Human-Mach Syst 52(6):1364–1374CrossRef

18.

Munawar A, Li Z, Nagururu N, Trakimas D, Kazanzides P, Taylor RH, Creighton FX (2023) Fully immersive virtual reality for skull-base surgery: surgical training and beyond. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-023-02956-5CrossRefPubMedPubMedCentral

19.

Shu H, Liang R, Li Z, Goodridge A, Zhang X, Ding H, Nagururu N, Sahu M, Creighton FX, Taylor RH et al (2023) Twin-s: a digital twin for skull base surgery. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-023-02863-9CrossRefPubMedPubMedCentral

20.

Zach C, Pock T, Bischof H (2007) A globally optimal algorithm for robust tv-l1 range image integration, pp. 1–8 . https://doi.org/10.1109/ICCV.2007.4408983

21.

Curless B, Levoy M (1996) A volumetric method for building complex models from range images. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques

22.

Lorensen W, Cline H (1987) Marching cubes: a high resolution 3d surface construction algorithm. ACM SIGGRAPH Computer Graphics 21:163CrossRef

23.

Zeng A, Song S, Nießner M, Fisher M, Xiao J, Funkhouser T (2017) 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In: CVPR

24.

Mangulabnan JE, Soberanis-Mukul RD, Teufel T, Hernáindez I, Winter J, Sahu M, Porras JL, Vedula SS, Ishii M, Hager G, Taylor RH, Unberath M (2023) A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy from Monocular Endoscopic Video

25.

Strobl K, Hirzinger G (2006) Optimal hand-eye calibration, pp. 4647–4653 . https://doi.org/10.1109/IROS.2006.282250

26.

Zhou Q-Y, Park J, Koltun V (2018) Open3d: A modern library for 3d data processing. arXiv:1801.09847

Title: An endoscopic chisel: intraoperative imaging carves 3D anatomical models
Authors: Jan Emily Mangulabnan
Roger D. Soberanis-Mukul
Timo Teufel
Manish Sahu
Jose L. Porras
S. Swaroop Vedula
Masaru Ishii
Gregory Hager
Russell H. Taylor
Mathias Unberath
Publication date: 16-05-2024
Publisher: Springer International Publishing
Keyword: Sinus Surgery
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 7/2024
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-024-03151-w

ERS Congress 2024

Springer Medicine

An endoscopic chisel: intraoperative imaging carves 3D anatomical models

Abstract

Purpose

Methods:

Results:

Conclusion:

ERS Congress 2024

Springer Medicine

Abstract

Purpose

Methods:

Results:

Conclusion:

Please log in to get access to this content

Other articles of this Issue 7/2024

Beyond the manual touch: situational-aware force control for increased safety in robot-assisted skullbase surgery

Structured light for touchless 3D registration in video-based surgical navigation

Head motion-corrected eye gaze tracking with the da Vinci surgical system

OneSLAM to map them all: a generalized approach to SLAM for monocular endoscopic imaging based on tracking any point

Enhancing surgical instrument segmentation: integrating vision transformer insights with adapter

Enabling extracorporeal ultrasound imaging with the da Vinci robot for transoral robotic surgery: a feasibility study