Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 6/2015

01-06-2015 | Original Article

Robust camera localisation with depth reconstruction for bronchoscopic navigation

Authors: Mali Shen, Stamatia Giannarou, Guang-Zhong Yang

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 6/2015

Login to get access

Abstract

Purpose

Bronchoscopy is a standard technique for airway examination, providing a minimally invasive approach for both diagnosis and treatment of pulmonary diseases. To target lesions identified pre-operatively, it is necessary to register the location of the bronchoscope to the CT bronchial model during the examination. Existing vision-based techniques rely on the registration between virtually rendered endobronchial images and videos based on image intensity or surface geometry. However, intensity-based approaches are sensitive to illumination artefacts, while gradient-based approaches are vulnerable to surface texture.

Methods

In this paper, depth information is employed in a novel way to achieve continuous and robust camera localisation. Surface shading has been used to recover depth from endobronchial images. The pose of the bronchoscopic camera is estimated by maximising the similarity between the depth recovered from a video image and that captured from a virtual camera projection of the CT model. The normalised cross-correlation and mutual information have both been used and compared for the similarity measure.

Results

The proposed depth-based tracking approach has been validated on both phantom and in vivo data. It outperforms the existing vision-based registration methods resulting in smaller pose estimation error of the bronchoscopic camera. It is shown that the proposed approach is more robust to illumination artefacts and surface texture and less sensitive to camera pose initialisation.

Conclusions

A reliable camera localisation technique has been proposed based on depth information for bronchoscopic navigation. Qualitative and quantitative performance evaluations show the clinical value of the proposed framework.
Literature
1.
go back to reference Becker HD, Herth F, Ernst A, Schwarz Y (2005) Bronchoscopic biopsy of peripheral lung lesions under electromagnetic guidance: a pilot study. J Bronchol Interv Pulmonol 12(1):9–13 Becker HD, Herth F, Ernst A, Schwarz Y (2005) Bronchoscopic biopsy of peripheral lung lesions under electromagnetic guidance: a pilot study. J Bronchol Interv Pulmonol 12(1):9–13
2.
go back to reference Brent RP (1973) Algorithms for minimization without derivatives. Prentice-Hall, Englewood Cliffs, NJ, USA Brent RP (1973) Algorithms for minimization without derivatives. Prentice-Hall, Englewood Cliffs, NJ, USA
3.
go back to reference Bricault I, Ferretti G, Cinquin P (1998) Registration of real and ct-derived virtual bronchoscopic images to assist transbronchial biopsy. IEEE Trans Biomed Eng 17(5):703–714 Bricault I, Ferretti G, Cinquin P (1998) Registration of real and ct-derived virtual bronchoscopic images to assist transbronchial biopsy. IEEE Trans Biomed Eng 17(5):703–714
4.
go back to reference Chung AJ, Deligianni F, Shah P, Wells A, Yang GZ (2006) Patient-specific bronchoscopy visualization through BRDF estimation and disocclusion correction. IEEE Trans Biomed Eng 25(4):503–513 Chung AJ, Deligianni F, Shah P, Wells A, Yang GZ (2006) Patient-specific bronchoscopy visualization through BRDF estimation and disocclusion correction. IEEE Trans Biomed Eng 25(4):503–513
5.
go back to reference Deguchi D, Mori K, Feuerstein M, Kitasaka T, Maurer CR Jr, Suenaga Y, Takabatake H, Mori M, Natori H (2009) Selective image similarity measure for bronchoscope tracking based on image registration. Med Image Anal 13(4):621–633CrossRefPubMed Deguchi D, Mori K, Feuerstein M, Kitasaka T, Maurer CR Jr, Suenaga Y, Takabatake H, Mori M, Natori H (2009) Selective image similarity measure for bronchoscope tracking based on image registration. Med Image Anal 13(4):621–633CrossRefPubMed
6.
go back to reference Deligianni F, Chung A, Yang GZ (2003) pq-space based 2d/3d registration for endoscope tracking. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2003, Springer, Berlin, Heidelberg, pp 311–318 Deligianni F, Chung A, Yang GZ (2003) pq-space based 2d/3d registration for endoscope tracking. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2003, Springer, Berlin, Heidelberg, pp 311–318
7.
go back to reference Deligianni F, Chung A, Yang GZ (2004) Patient-specific bronchoscope simulation with pq-space-based 2d/3d registration. Comput Aided Surg 9(5):215–226PubMed Deligianni F, Chung A, Yang GZ (2004) Patient-specific bronchoscope simulation with pq-space-based 2d/3d registration. Comput Aided Surg 9(5):215–226PubMed
8.
go back to reference Deligianni F, Chung AJ, Yang GZ (2006) Nonrigid 2-d/3-d registration for patient specific bronchoscopy simulation with statistical shape modeling: phantom validation. IEEE Trans Biomed Eng 25(11):1462–1471 Deligianni F, Chung AJ, Yang GZ (2006) Nonrigid 2-d/3-d registration for patient specific bronchoscopy simulation with statistical shape modeling: phantom validation. IEEE Trans Biomed Eng 25(11):1462–1471
9.
go back to reference Higgins WE, Helferty JP, Padfield DR (2003) Integrated bronchoscopic video tracking and 3d ct registration for virtual bronchoscopy. In: SPIE Medical Imaging, International Society for Optics and Photonics, pp 80–89 Higgins WE, Helferty JP, Padfield DR (2003) Integrated bronchoscopic video tracking and 3d ct registration for virtual bronchoscopy. In: SPIE Medical Imaging, International Society for Optics and Photonics, pp 80–89
10.
go back to reference Horn BKP (1986) Robot vision. MIT press, Cambridge, MA, USA Horn BKP (1986) Robot vision. MIT press, Cambridge, MA, USA
11.
go back to reference Kao CY, Osher S, Tsai YH (2005) Fast sweeping methods for static Hamilton–Jacobi equations. SIAM J Numer Anal 42(6):2612–2632CrossRef Kao CY, Osher S, Tsai YH (2005) Fast sweeping methods for static Hamilton–Jacobi equations. SIAM J Numer Anal 42(6):2612–2632CrossRef
12.
go back to reference Lei Y, Jiu-Qiang H (2008) A perspective shape-from-shading method using fast sweeping numerical scheme. Opt Appl 38(2):387–398 Lei Y, Jiu-Qiang H (2008) A perspective shape-from-shading method using fast sweeping numerical scheme. Opt Appl 38(2):387–398
13.
go back to reference Luo X, Feuerstein M, Sugiura T, Kitasaka T, Imaizumi K, Hasegawa Y, Mori K (2010) Towards hybrid bronchoscope tracking under respiratory motion: evaluation on a dynamic motion phantom. In: SPIE Medical Imaging, International Society for Optics and Photonics, p 76251B Luo X, Feuerstein M, Sugiura T, Kitasaka T, Imaizumi K, Hasegawa Y, Mori K (2010) Towards hybrid bronchoscope tracking under respiratory motion: evaluation on a dynamic motion phantom. In: SPIE Medical Imaging, International Society for Optics and Photonics, p 76251B
14.
go back to reference Luo X, Jayarathne U, McLeod A, Mori K (2014) Enhanced differential evolution to combine optical mouse sensor with image structural patches for robust endoscopic navigation. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2014, Springer International Publishing, pp 340–348 Luo X, Jayarathne U, McLeod A, Mori K (2014) Enhanced differential evolution to combine optical mouse sensor with image structural patches for robust endoscopic navigation. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2014, Springer International Publishing, pp 340–348
15.
go back to reference Mori K, Enjoji T, Deguchi D, Kitasaka T, Suenaga Y, Toriwaki J, Takabatake H, Natori H (2004) New image similarity measures for bronchoscope tracking based on image registration between virtual and real bronchoscopic images. In: SPIE Medical Imaging, International Society for Optics and Photonics, pp 165–176 Mori K, Enjoji T, Deguchi D, Kitasaka T, Suenaga Y, Toriwaki J, Takabatake H, Natori H (2004) New image similarity measures for bronchoscope tracking based on image registration between virtual and real bronchoscopic images. In: SPIE Medical Imaging, International Society for Optics and Photonics, pp 165–176
16.
go back to reference Mori K, Ishitani K, Deguchi D, Kitasaka T, Suenaga Y, Takabatake H, Mori M, Natori H (2007) Compensation of electromagnetic tracking system using an optical tracker and its application to bronchoscopy navigation system. In: SPIE Medical Imaging, International Society for Optics and Photonics, p 65090M Mori K, Ishitani K, Deguchi D, Kitasaka T, Suenaga Y, Takabatake H, Mori M, Natori H (2007) Compensation of electromagnetic tracking system using an optical tracker and its application to bronchoscopy navigation system. In: SPIE Medical Imaging, International Society for Optics and Photonics, p 65090M
17.
go back to reference Nagao J, Mori K, Enjouji T, Deguchi D, Kitasaka T, Suenaga Y, Hasegawa J, Toriwaki J, Takabatake H, Natori H (2004) Fast and accurate bronchoscope tracking using image registration and motion prediction. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2004, Springer, Berlin, Heidelberg, pp 551–558 Nagao J, Mori K, Enjouji T, Deguchi D, Kitasaka T, Suenaga Y, Hasegawa J, Toriwaki J, Takabatake H, Natori H (2004) Fast and accurate bronchoscope tracking using image registration and motion prediction. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2004, Springer, Berlin, Heidelberg, pp 551–558
18.
go back to reference Powell MJD (1964) An efficient method for finding the minimum of a function of several variables without calculating derivatives. Comput J 7(2):155–162CrossRef Powell MJD (1964) An efficient method for finding the minimum of a function of several variables without calculating derivatives. Comput J 7(2):155–162CrossRef
19.
go back to reference Prados E, Faugeras O (2005) Shape from shading: a well-posed problem? In: IEEE Conference on Computer Vision and Pattern Recognition, pp 870–877 Prados E, Faugeras O (2005) Shape from shading: a well-posed problem? In: IEEE Conference on Computer Vision and Pattern Recognition, pp 870–877
20.
go back to reference Rai L, Helferty JP, Higgins WE (2008) Combined video tracking and image-video registration for continuous bronchoscopic guidance. Int J Comput Assist Radiol Surg 3(3–4):315–329 Rai L, Helferty JP, Higgins WE (2008) Combined video tracking and image-video registration for continuous bronchoscopic guidance. Int J Comput Assist Radiol Surg 3(3–4):315–329
21.
go back to reference Rashid HU, Burger P (1992) Differential algorithm for the determination of shape from shading using a point light source. Image Vis Comput 10(2):119–127 Rashid HU, Burger P (1992) Differential algorithm for the determination of shape from shading using a point light source. Image Vis Comput 10(2):119–127
22.
go back to reference Shah PL, Herth FJ (2013) Current status of bronchoscopic lung volume reduction with endobronchial valves. Thorax 69:280–286CrossRefPubMed Shah PL, Herth FJ (2013) Current status of bronchoscopic lung volume reduction with endobronchial valves. Thorax 69:280–286CrossRefPubMed
23.
go back to reference Solomon SB, White P Jr, Wiener CM, Orens JB, Wang KP (2000) Three-dimensional ct-guided bronchoscopy with a real-time electromagnetic position sensor: a comparison of two image registration methods. CHEST J 118(6):1783–1787 Solomon SB, White P Jr, Wiener CM, Orens JB, Wang KP (2000) Three-dimensional ct-guided bronchoscopy with a real-time electromagnetic position sensor: a comparison of two image registration methods. CHEST J 118(6):1783–1787
24.
go back to reference Soper TD, Haynor DR, Glenny RW, Seibel EJ (2010) In vivo validation of a hybrid tracking system for navigation of an ultrathin bronchoscope within peripheral airways. IEEE Trans Biomed Eng 57(3):736–745CrossRefPubMed Soper TD, Haynor DR, Glenny RW, Seibel EJ (2010) In vivo validation of a hybrid tracking system for navigation of an ultrathin bronchoscope within peripheral airways. IEEE Trans Biomed Eng 57(3):736–745CrossRefPubMed
25.
go back to reference Visentini-Scarzanella M, Stoyanov D, Yang GZ (2012) Metric depth recovery from monocular images using shape-from-shading and specularities. In: 19th IEEE International Conference on Image Processing (ICIP), pp 25–28 Visentini-Scarzanella M, Stoyanov D, Yang GZ (2012) Metric depth recovery from monocular images using shape-from-shading and specularities. In: 19th IEEE International Conference on Image Processing (ICIP), pp 25–28
26.
go back to reference Zhang L, Yip AM, Brown MS, Tan CL (2009) A unified framework for document restoration using inpainting and shape-from-shading. Pattern Recognit 42(11):2961–2978CrossRef Zhang L, Yip AM, Brown MS, Tan CL (2009) A unified framework for document restoration using inpainting and shape-from-shading. Pattern Recognit 42(11):2961–2978CrossRef
Metadata
Title
Robust camera localisation with depth reconstruction for bronchoscopic navigation
Authors
Mali Shen
Stamatia Giannarou
Guang-Zhong Yang
Publication date
01-06-2015
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 6/2015
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-015-1197-y

Other articles of this Issue 6/2015

International Journal of Computer Assisted Radiology and Surgery 6/2015 Go to the issue

Editorial

Preface