Skip to main content
Top
Published in: Surgical Endoscopy 12/2012

01-12-2012

Real-time three-dimensional soft tissue reconstruction for laparoscopic surgery

Authors: Jędrzej Kowalczuk, Avishai Meyer, Jay Carlson, Eric T. Psota, Shelby Buettner, Lance C. Pérez, Shane M. Farritor, Dmitry Oleynikov

Published in: Surgical Endoscopy | Issue 12/2012

Login to get access

Abstract

Background

Accurate real-time 3D models of the operating field have the potential to enable augmented reality for endoscopic surgery. A new system is proposed to create real-time 3D models of the operating field that uses a custom miniaturized stereoscopic video camera attached to a laparoscope and an image-based reconstruction algorithm implemented on a graphics processing unit (GPU).

Methods

The proposed system was evaluated in a porcine model that approximates the viewing conditions of in vivo surgery. To assess the quality of the models, a synthetic view of the operating field was produced by overlaying a color image on the reconstructed 3D model, and an image rendered from the 3D model was compared with a 2D image captured from the same view.

Results

Experiments conducted with an object of known geometry demonstrate that the system produces 3D models accurate to within 1.5 mm.

Conclusions

The ability to produce accurate real-time 3D models of the operating field is a significant advancement toward augmented reality in minimally invasive surgery. An imaging system with this capability will potentially transform surgery by helping novice and expert surgeons alike to delineate variance in internal anatomy accurately.
Literature
1.
go back to reference Devernay F (2001) 3D Reconstruction of the operating field for image overlay in 3D-endoscopic surgery. In: Proceedings of the IEEE and ACM international symposium on augmented reality (ISAR’01), Washington, DC, pp 191–193 Devernay F (2001) 3D Reconstruction of the operating field for image overlay in 3D-endoscopic surgery. In: Proceedings of the IEEE and ACM international symposium on augmented reality (ISAR’01), Washington, DC, pp 191–193
2.
go back to reference Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD (2009) Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73:896–900PubMedCrossRef Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD (2009) Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73:896–900PubMedCrossRef
3.
go back to reference Bethea B, Okamura A, Kitagawa M, Fitton T, Cattaneo S, Gott V, Baumgartner D, Yuh WA (2004) Application of haptic feedback to robotic surgery. J Laparoendosc Adv Surg Tech 14:191–195CrossRef Bethea B, Okamura A, Kitagawa M, Fitton T, Cattaneo S, Gott V, Baumgartner D, Yuh WA (2004) Application of haptic feedback to robotic surgery. J Laparoendosc Adv Surg Tech 14:191–195CrossRef
4.
go back to reference Stoyanov D, Darzi A, Yang GZ (2005) A practical approach towards accurate dense 3D depth recovery for robotic laparoscopic surgery. Comput Aided Surg 10:199–208PubMed Stoyanov D, Darzi A, Yang GZ (2005) A practical approach towards accurate dense 3D depth recovery for robotic laparoscopic surgery. Comput Aided Surg 10:199–208PubMed
5.
go back to reference Cano González AM, Sánchez-González P, Sánchez-Margallo FM, Oropesa I, Pozo F, Gómez EJ (2009) Video-endoscopic image analysis for 3D reconstruction of the surgical scene. In: 4th European conference of the international federation for medical and biological engineering, vol 22, Antwerp, Belgium, pp 923–926 Cano González AM, Sánchez-González P, Sánchez-Margallo FM, Oropesa I, Pozo F, Gómez EJ (2009) Video-endoscopic image analysis for 3D reconstruction of the surgical scene. In: 4th European conference of the international federation for medical and biological engineering, vol 22, Antwerp, Belgium, pp 923–926
6.
go back to reference Stoyanov D, Scarzanella M, Pratt P, Yang GZ (2010) Real-time stereo reconstruction in robotically assisted minimally invasive surgery. Med Image Comput Comput Assist Interv 13:275–282PubMed Stoyanov D, Scarzanella M, Pratt P, Yang GZ (2010) Real-time stereo reconstruction in robotically assisted minimally invasive surgery. Med Image Comput Comput Assist Interv 13:275–282PubMed
7.
go back to reference Taylor R, Stoianovici D (2003) Medical robotics in computer-integrated surgery. IEEE Trans Robot Autom 19:765–781CrossRef Taylor R, Stoianovici D (2003) Medical robotics in computer-integrated surgery. IEEE Trans Robot Autom 19:765–781CrossRef
8.
go back to reference Hu M, Penney G, Rueckert D, Edwards P, Bello F, Casula R, Figl M, Hawkes D (2009) Nonrigid reconstruction of the beating heart surface for minimally invasive cardiac surgery. In: Proceedings of medical image computing and computer-assisted, London, pp 34–42 Hu M, Penney G, Rueckert D, Edwards P, Bello F, Casula R, Figl M, Hawkes D (2009) Nonrigid reconstruction of the beating heart surface for minimally invasive cardiac surgery. In: Proceedings of medical image computing and computer-assisted, London, pp 34–42
9.
go back to reference Figl M, Rueckert D, Hawkes D, Casula R, Hu M, Pedro O, Zhang DP, Penney D, Bello F, Edwards P (2010) Image guidance for robotic minimally invasive coronary artery bypass. Comput Med Imaging Graphics 34:61–68CrossRef Figl M, Rueckert D, Hawkes D, Casula R, Hu M, Pedro O, Zhang DP, Penney D, Bello F, Edwards P (2010) Image guidance for robotic minimally invasive coronary artery bypass. Comput Med Imaging Graphics 34:61–68CrossRef
10.
go back to reference Stoyanov D, Darzi A, Yang G (2004) Dense 3D depth recovery for soft tissue deformation during robotically assisted laparoscopic surgery. In: Proceedings of medical image computing and computer-assisted intervention, Springer, Heidelberg, pp 41–48 Stoyanov D, Darzi A, Yang G (2004) Dense 3D depth recovery for soft tissue deformation during robotically assisted laparoscopic surgery. In: Proceedings of medical image computing and computer-assisted intervention, Springer, Heidelberg, pp 41–48
11.
go back to reference Lo B, Scarzanella M, Stoyanov D, Yang GZ (2008) Belief propagation for depth cue fusion in minimally invasive surgery. MICCAI 2:104–112 Lo B, Scarzanella M, Stoyanov D, Yang GZ (2008) Belief propagation for depth cue fusion in minimally invasive surgery. MICCAI 2:104–112
12.
go back to reference Psota ET, Kowalczuk J, Carlson J, Pérez LC (2011) A local iterative refinement method for adaptive support-weight stereo matching. In: International conference on image processing, computer vision and pattern recognition (IPCV), Las Vegas, pp 271–277 Psota ET, Kowalczuk J, Carlson J, Pérez LC (2011) A local iterative refinement method for adaptive support-weight stereo matching. In: International conference on image processing, computer vision and pattern recognition (IPCV), Las Vegas, pp 271–277
Metadata
Title
Real-time three-dimensional soft tissue reconstruction for laparoscopic surgery
Authors
Jędrzej Kowalczuk
Avishai Meyer
Jay Carlson
Eric T. Psota
Shelby Buettner
Lance C. Pérez
Shane M. Farritor
Dmitry Oleynikov
Publication date
01-12-2012
Publisher
Springer-Verlag
Published in
Surgical Endoscopy / Issue 12/2012
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-012-2355-8

Other articles of this Issue 12/2012

Surgical Endoscopy 12/2012 Go to the issue