Skip to main content
Top
Published in: Surgical Endoscopy 8/2014

01-08-2014 | Video

Robotic duodenopancreatectomy assisted with augmented reality and real-time fluorescence guidance

Authors: Patrick Pessaux, Michele Diana, Luc Soler, Tullio Piardi, Didier Mutter, Jacques Marescaux

Published in: Surgical Endoscopy | Issue 8/2014

Login to get access

Abstract

Background

The minimally invasive surgeon cannot use ‘sense of touch’ to orientate surgical resection, identifying important structures (vessels, tumors, etc.) by manual palpation. Robotic research has provided technology to facilitate laparoscopic surgery; however, robotics has yet to solve the lack of tactile feedback inherent to keyhole surgery. Misinterpretation of the vascular supply and tumor location may increase the risk of intraoperative bleeding and worsen dissection with positive resection margins.

Methods

Augmented reality (AR) consists of the fusion of synthetic computer-generated images (three-dimensional virtual model) obtained from medical imaging preoperative work-up and real-time patient images with the aim of visualizing unapparent anatomical details.

Results

In this article, we review the most common modalities used to achieve surgical navigation through AR, along with a report of a case of robotic duodenopancreatectomy using AR guidance complemented with the use of fluorescence guidance.

Conclusions

The presentation of this complex and high-technology case of robotic duodenopancreatectomy, and the overview of current technology that has made it possible to use AR in the operating room, highlights the needs for further evolution and the windows of opportunity to create a new paradigm in surgical practice.
Appendix
Available only for authorised users
Literature
1.
go back to reference Tekkis PP, Senagore AJ, Delaney CP, Fazio VW (2005) Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right-sided and left-sided resections. Ann Surg 242:83–91PubMedCentralPubMedCrossRef Tekkis PP, Senagore AJ, Delaney CP, Fazio VW (2005) Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right-sided and left-sided resections. Ann Surg 242:83–91PubMedCentralPubMedCrossRef
2.
go back to reference Diana M, Pessaux P, Marescaux J (2014) New technologies for single-site robotic surgery in hepato-biliary-pancreatic surgery. J Hepatobiliary Pancreat Sci 21:34–42PubMedCrossRef Diana M, Pessaux P, Marescaux J (2014) New technologies for single-site robotic surgery in hepato-biliary-pancreatic surgery. J Hepatobiliary Pancreat Sci 21:34–42PubMedCrossRef
3.
go back to reference Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20:189–201PubMedCrossRef Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20:189–201PubMedCrossRef
4.
go back to reference D’Agostino J, Diana M, Soler L, Vix M, Marescaux J (2012) 3D virtual neck exploration prior to parathyroidectomy. N Engl J Med 367(11):1072–1073PubMedCrossRef D’Agostino J, Diana M, Soler L, Vix M, Marescaux J (2012) 3D virtual neck exploration prior to parathyroidectomy. N Engl J Med 367(11):1072–1073PubMedCrossRef
5.
go back to reference Iseki H, Masutani Y, Iwahara M, Tanikawa T, Muragaki Y, Taira T et al (1997) Volumegraph (overlaid three-dimensional image-guided navigation). Clinical application of augmented reality in neurosurgery. Stereotact Funct Neurosurg 68:18–24PubMedCrossRef Iseki H, Masutani Y, Iwahara M, Tanikawa T, Muragaki Y, Taira T et al (1997) Volumegraph (overlaid three-dimensional image-guided navigation). Clinical application of augmented reality in neurosurgery. Stereotact Funct Neurosurg 68:18–24PubMedCrossRef
6.
go back to reference Wagner A, Ploder O, Enislidis G, Truppe M, Ewers R (1995) Virtual image guided navigation in tumor surgery: technical innovation. J Craniomaxillofac Surg 23:217–223PubMedCrossRef Wagner A, Ploder O, Enislidis G, Truppe M, Ewers R (1995) Virtual image guided navigation in tumor surgery: technical innovation. J Craniomaxillofac Surg 23:217–223PubMedCrossRef
8.
go back to reference Calhoun PS, Kuszyk BS, Heath DG, Carley JC, Fishman EK (1999) Three-dimensional volume rendering of spiral CT data: theory and method. Radiographics 19:745–764PubMedCrossRef Calhoun PS, Kuszyk BS, Heath DG, Carley JC, Fishman EK (1999) Three-dimensional volume rendering of spiral CT data: theory and method. Radiographics 19:745–764PubMedCrossRef
9.
go back to reference Volonte F, Pugin F, Bucher P, Sugimoto M, Ratib O, Morel P (2011) Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci 18:506–509PubMedCrossRef Volonte F, Pugin F, Bucher P, Sugimoto M, Ratib O, Morel P (2011) Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci 18:506–509PubMedCrossRef
10.
go back to reference Pieper S, Halle M, Kikinis R (2004) 3D SLICER. In: Proceedings of the 1st IEEE International Symposium on Biomedical Imaging: from nano to macro, pp 632–635 Pieper S, Halle M, Kikinis R (2004) 3D SLICER. In: Proceedings of the 1st IEEE International Symposium on Biomedical Imaging: from nano to macro, pp 632–635
11.
go back to reference Fichtinger G, Deguet A, Fischer G, Iordachita I, Balogh E, Masamune K et al (2005) Image overlay for CT-guided needle insertions. Comput Aided Surg 10:241–255PubMedCrossRef Fichtinger G, Deguet A, Fischer G, Iordachita I, Balogh E, Masamune K et al (2005) Image overlay for CT-guided needle insertions. Comput Aided Surg 10:241–255PubMedCrossRef
12.
go back to reference Liao H, Inomata T, Sakuma I, Dohi T (2010) 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Biomed Eng 57:1476–1486PubMedCrossRef Liao H, Inomata T, Sakuma I, Dohi T (2010) 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Biomed Eng 57:1476–1486PubMedCrossRef
13.
go back to reference Okamoto T, Onda S, Matsumoto M, Gocho T, Futagawa Y, Fujioka S et al (2013) Utility of augmented reality system in hepatobiliary surgery. J Hepatobiliary Pancreat Sci 20(2):249–253PubMedCrossRef Okamoto T, Onda S, Matsumoto M, Gocho T, Futagawa Y, Fujioka S et al (2013) Utility of augmented reality system in hepatobiliary surgery. J Hepatobiliary Pancreat Sci 20(2):249–253PubMedCrossRef
14.
go back to reference Sugimoto M, Yasuda H, Koda K, Suzuki M, Yamazaki M, Tezuka T et al (2010) Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci 17:629–636PubMedCrossRef Sugimoto M, Yasuda H, Koda K, Suzuki M, Yamazaki M, Tezuka T et al (2010) Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci 17:629–636PubMedCrossRef
15.
go back to reference Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented-reality-assisted laparoscopic adrenalectomy. JAMA 292:2214–2215PubMed Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented-reality-assisted laparoscopic adrenalectomy. JAMA 292:2214–2215PubMed
16.
go back to reference Mutter D, Soler L, Marescaux J (2010) Recent advances in liver imaging. Expert Rev Gastroenterol Hepatol 4:613–621PubMedCrossRef Mutter D, Soler L, Marescaux J (2010) Recent advances in liver imaging. Expert Rev Gastroenterol Hepatol 4:613–621PubMedCrossRef
17.
go back to reference D’Agostino J, Wall J, Soler L, Vix M, Duh QY, Marescaux J (2013) Virtual neck exploration for parathyroid adenomas: a first step toward minimally invasive image-guided surgery. JAMA Surg 148:232–238 discussion 238PubMedCrossRef D’Agostino J, Wall J, Soler L, Vix M, Duh QY, Marescaux J (2013) Virtual neck exploration for parathyroid adenomas: a first step toward minimally invasive image-guided surgery. JAMA Surg 148:232–238 discussion 238PubMedCrossRef
18.
go back to reference Marzano E, Piardi T, Soler L, Diana M, Mutter D, Marescaux J et al (2013) Augmented reality-guided artery-first pancreatico-duodenectomy. J Gastrointest Surg 17:1980–1983PubMedCrossRef Marzano E, Piardi T, Soler L, Diana M, Mutter D, Marescaux J et al (2013) Augmented reality-guided artery-first pancreatico-duodenectomy. J Gastrointest Surg 17:1980–1983PubMedCrossRef
19.
go back to reference Marvik R, Lango T, Tangen GA, Andersen JO, Kaspersen JH, Ystgaard B et al (2004) Laparoscopic navigation pointer for three-dimensional image-guided surgery. Surg Endosc 18:1242–1248PubMedCrossRef Marvik R, Lango T, Tangen GA, Andersen JO, Kaspersen JH, Ystgaard B et al (2004) Laparoscopic navigation pointer for three-dimensional image-guided surgery. Surg Endosc 18:1242–1248PubMedCrossRef
20.
go back to reference Ieiri S, Uemura M, Konishi K, Souzaki R, Nagao Y, Tsutsumi N et al (2012) Augmented reality navigation system for laparoscopic splenectomy in children based on preoperative CT image using optical tracking device. Pediatr Surg Int 28:341–346PubMedCrossRef Ieiri S, Uemura M, Konishi K, Souzaki R, Nagao Y, Tsutsumi N et al (2012) Augmented reality navigation system for laparoscopic splenectomy in children based on preoperative CT image using optical tracking device. Pediatr Surg Int 28:341–346PubMedCrossRef
21.
go back to reference Nam WH, Kang DG, Lee D, Lee JY, Ra JB (2012) Automatic registration between 3D intra-operative ultrasound and pre-operative CT images of the liver based on robust edge matching. Phys Med Biol 57:69–91PubMedCrossRef Nam WH, Kang DG, Lee D, Lee JY, Ra JB (2012) Automatic registration between 3D intra-operative ultrasound and pre-operative CT images of the liver based on robust edge matching. Phys Med Biol 57:69–91PubMedCrossRef
22.
go back to reference Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C et al (2010) Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24:1976–1985PubMedCrossRef Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C et al (2010) Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24:1976–1985PubMedCrossRef
23.
go back to reference Hostettler A, Nicolau SA, Remond Y, Marescaux J, Soler L (2010) A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Prog Biophys Mol Biol 103:169–184PubMedCrossRef Hostettler A, Nicolau SA, Remond Y, Marescaux J, Soler L (2010) A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Prog Biophys Mol Biol 103:169–184PubMedCrossRef
24.
go back to reference Urbanavicius L, Pattyn P, de Putte DV, Venskutonis D (2011) How to assess intestinal viability during surgery: a review of techniques. World J Gastrointest Surg 3:59–69PubMedCentralPubMedCrossRef Urbanavicius L, Pattyn P, de Putte DV, Venskutonis D (2011) How to assess intestinal viability during surgery: a review of techniques. World J Gastrointest Surg 3:59–69PubMedCentralPubMedCrossRef
25.
go back to reference Diana M, Noll E, Diemunsch P, Dallemagne B, Benahmed MA, Agnus V, et al (2013) Enhanced-reality video fluorescence: a real-time assessment of intestinal viability. Ann Surg Diana M, Noll E, Diemunsch P, Dallemagne B, Benahmed MA, Agnus V, et al (2013) Enhanced-reality video fluorescence: a real-time assessment of intestinal viability. Ann Surg
Metadata
Title
Robotic duodenopancreatectomy assisted with augmented reality and real-time fluorescence guidance
Authors
Patrick Pessaux
Michele Diana
Luc Soler
Tullio Piardi
Didier Mutter
Jacques Marescaux
Publication date
01-08-2014
Publisher
Springer US
Published in
Surgical Endoscopy / Issue 8/2014
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-014-3465-2

Other articles of this Issue 8/2014

Surgical Endoscopy 8/2014 Go to the issue