Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-12-2015 | Original Article

Robust surface tracking combining features, intensity and illumination compensation

Authors: Xiaofei Du, Neil Clancy, Shobhit Arya, George B. Hanna, John Kelly, Daniel S. Elson, Danail Stoyanov

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 12/2015

Abstract

Purpose

Recovering tissue deformation during robotic-assisted minimally invasive surgery procedures is important for providing intra-operative guidance, enabling in vivo imaging modalities and enhanced robotic control. The tissue motion can also be used to apply motion stabilization and to prescribe dynamic constraints for avoiding critical anatomical structures.

Methods

Image-based methods based independently on salient features or on image intensity have limitations when dealing with homogeneous soft tissues or complex reflectance. In this paper, we use a triangular geometric mesh model in order to combine the advantages of both feature and intensity information and track the tissue surface reliably and robustly.

Results

Synthetic and in vivo experiments are performed to provide quantitative analysis of the tracking accuracy of our method, and we also show exemplar results for registering multispectral images where there is only a weak image signal.

Conclusion

Compared to traditional methods, our hybrid tracking method is more robust and has improved convergence in the presence of larger displacements, tissue dynamics and illumination changes.

Available only for authorised users

http://www.surgicalvision.cs.ucl.ac.uk/data.

Baker S, Matthews I (2004) Lucas–Kanade 20 years on: a unifying framework. Int J Comput Vis 56(3):221–255CrossRef

Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. In: Computer Vision-ECCV 2006, Springer, Berlin, Heidelberg, pp 404–417

Bradski G (2000) The opencv library. Doctor Dobbs Journal 25(11):120–126

Braux-Zin J, Dupont R, Bartoli A (2013) Combining features and intensity for wide-baseline non-rigid surface registration. In: British machine vision conference (BMVC), BMVA

Clancy NT, Stoyanov D, James DR, Di Marco A, Sauvage V, Clark J, Yang GZ, Elson DS (2012) Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery. Biomed Opt Exp 3(10):2567–2578CrossRef

Delabarre B, Marchand E (2012) Visual servoing using the sum of conditional variance. In: Intelligent robots and systems (IROS), 2012 IEEE/RSJ international conference on, IEEE, pp 1689–1694

Fua P, Brechbühler C (1996) Imposing hard constraints on soft snakes. In: Computer vision-ECCV 1996, Springer, pp 495– 506

Giannarou S, Visentini-Scarzanella M, Yang GZ (2009) Affine-invariant anisotropic detector for soft tissue tracking in minimally invasive surgery. In: Biomedical imaging: from Nano to Macro, 2009. ISBI’09. IEEE international symposium on, IEEE, pp 1059–1062

Giannarou S, Visentini-Scarzanella M, Yang GZ (2013) Probabilistic tracking of affine-invariant anisotropic regions. Pattern Anal Mach Intell IEEE Trans 35(1):130–143CrossRef

10.

Gröger M, Sepp W, Ortmaier T, Hirzinger G (2001) Reconstruction of image structure in presence of specular reflections. In: Pattern recognition, Springer, pp 53–60

11.

Kalal Z, Mikolajczyk K, Matas J (2010) Forward–backward error: automatic detection of tracking failures. In: Pattern recognition (ICPR), 2010 20th international conference on, IEEE, pp 2756–2759

12.

Keerthi SS, DeCoste D (2005) A modified finite newton method for fast solution of large scale linear svms. J Mach Learn Res 6:341–361

13.

Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D (2013) Optical techniques for 3d surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal 17(8):974–996CrossRefPubMed

14.

Mangasarian OL (2002) A finite newton method for classification. Opt Method Softw 17(5):913–929CrossRef

15.

Matthews I, Ishikawa T, Baker S (2004) The template update problem. Pattern Anal Mach Intell IEEE Trans 26(6):810–815CrossRef

16.

Mountney P, Yang GZ (2008) Soft tissue tracking for minimally invasive surgery: learning local deformation online. In: Medical image computing and computer-assisted intervention-MICCAI 2008, Springer, pp 364–372

17.

Ortmaier TJ (2003) Motion compensation in minimally invasive robotic surgery. PhD thesis, Universität München

18.

Pickering MR, Muhit AA, Scarvell JM, Smith PN (2009) A new multi-modal similarity measure for fast gradient-based 2d–3d image registration. In: Engineering in medicine and biology society, 2009. EMBC 2009, annual international conference of the IEEE, IEEE, pp 5821–5824

19.

Pilet J, Lepetit V, Fua P (2008) Fast non-rigid surface detection, registration and realistic augmentation. Int J Comput Vis 76(2):109–122CrossRef

20.

Puerto-Souza GA, Mariottini GL (2012) Hierarchical multi-affine (hma) algorithm for fast and accurate feature matching in minimally-invasive surgical images. In: Intelligent robots and systems (IROS), 2012 IEEE/RSJ international conference on, IEEE, pp 2007–2012

21.

Richa R, Sznitman R, Taylor R, Hager G (2011) Visual tracking using the sum of conditional variance. In: Intelligent robots and systems (IROS), 2011 IEEE/RSJ international conference on, IEEE, pp 2953–2958

22.

Selka F, Nicolau SA, Agnus V, Bessaid A, Marescaux J, Soler L (2013) Evaluation of endoscopic image enhancement for feature tracking: a new validation framework. In: Augmented reality environments for medical imaging and computer-assisted interventions, Springer, pp 75–85

23.

Stoyanov D (2012) Stereoscopic scene flow for robotic assisted minimally invasive surgery. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2012, Springer, pp 479–486

24.

Stoyanov D (2012) Surgical vision. Ann Biomed Eng 40(2):332–345CrossRefPubMed

25.

Stoyanov D, Yang GZ (2005) Removing specular reflection components for robotic assisted laparoscopic surgery. In: Image processing, 2005. ICIP 2005. IEEE international conference on, IEEE, vol 3, pp III-632

26.

Stoyanov D, Darzi A, Yang GZ (2004) Dense 3d depth recovery for soft tissue deformation during robotically assisted laparoscopic surgery. In: Medical image computing and computer-assisted intervention-MICCAI 2004, Springer, pp 41–48

27.

Stoyanov D, Darzi A, Yang GZ (2005) A practical approach towards accurate dense 3d depth recovery for robotic laparoscopic surgery. Comput Aided Surg 10(4):199–208

28.

Stoyanov D, Rayshubskiy A, Hillman E (2012) Robust registration of multispectral images of the cortical surface in neurosurgery. In: Biomedical imaging (ISBI), 2012 9th IEEE international symposium on, IEEE, pp 1643–1646

29.

Yip MC, Lowe DG, Salcudean SE, Rohling RN, Nguan CY (2012) Real-time methods for long-term tissue feature tracking in endoscopic scenes. In: Information processing in computer-assisted interventions, Springer, pp 33–43

30.

Zhu J, Lyu MR, Huang TS (2009) A fast 2d shape recovery approach by fusing features and appearance. Pattern Anal Mach Intell IEEE Trans 31(7):1210–1224CrossRef

Title: Robust surface tracking combining features, intensity and illumination compensation
Authors: Xiaofei Du
Neil Clancy
Shobhit Arya
George B. Hanna
John Kelly
Daniel S. Elson
Danail Stoyanov
Publication date: 01-12-2015
Publisher: Springer Berlin Heidelberg
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 12/2015
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-015-1243-9

At a glance: The STEP trials

Springer Medicine

Robust surface tracking combining features, intensity and illumination compensation

Abstract

Purpose

Methods

Results

Conclusion

At a glance: The STEP trials

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 12/2015

Multimodal image-guided prostate fusion biopsy based on automatic deformable registration

Statistical study of parameters for deep brain stimulation automatic preoperative planning of electrodes trajectories

Robot guidance of an ultrasound probe toward a 3D region of interest detected through X-ray mammography

An endoscopic structured light system using multispectral detection

Intraoperative CT as a registration benchmark for intervertebral motion compensation in image-guided open spinal surgery

Deep venous thrombosis identification from analysis of ultrasound data