Skip to main content
Top
Published in: Surgical Endoscopy 8/2010

01-08-2010

Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography

Authors: Raj Shekhar, Omkar Dandekar, Venkatesh Bhat, Mathew Philip, Peng Lei, Carlos Godinez, Erica Sutton, Ivan George, Steven Kavic, Reuben Mezrich, Adrian Park

Published in: Surgical Endoscopy | Issue 8/2010

Login to get access

Abstract

Background

Current laparoscopic images are rich in surface detail but lack information on deeper structures. This report presents a novel method for highlighting these structures during laparoscopic surgery using continuous multislice computed tomography (CT). This has resulted in a more accurate augmented reality (AR) approach, termed “live AR,” which merges three-dimensional (3D) anatomy from live low-dose intraoperative CT with live images from the laparoscope.

Methods

A series of procedures with swine was conducted in a CT room with a fully equipped laparoscopic surgical suite. A 64-slice CT scanner was used to image the surgical field approximately once per second. The procedures began with a contrast-enhanced, diagnostic-quality CT scan (initial CT) of the liver followed by continuous intraoperative CT and laparoscopic imaging with an optically tracked laparoscope. Intraoperative anatomic changes included user-applied deformations and those from breathing. Through deformable image registration, an intermediate image processing step, the initial CT was warped to align spatially with the low-dose intraoperative CT scans. The registered initial CT then was rendered and merged with laparoscopic images to create live AR.

Results

Superior compensation for soft tissue deformations using the described method led to more accurate spatial registration between laparoscopic and rendered CT images with live AR than with conventional AR. Moreover, substitution of low-dose CT with registered initial CT helped with continuous visualization of the vasculature and offered the potential of at least an eightfold reduction in intraoperative X-ray dose.

Conclusions

The authors proposed and developed live AR, a new surgical visualization approach that merges rich surface detail from a laparoscope with instantaneous 3D anatomy from continuous CT scanning of the surgical field. Through innovative use of deformable image registration, they also demonstrated the feasibility of continuous visualization of the vasculature and considerable X-ray dose reduction. This study provides motivation for further investigation and development of live AR.
Appendix
Available only for authorised users
Literature
3.
go back to reference Osborne DA, Alexander G, Boe B, Zervos EE (2006) Laparoscopic cholecystectomy: past, present, and future. Surg Technol Int 15:81–85PubMed Osborne DA, Alexander G, Boe B, Zervos EE (2006) Laparoscopic cholecystectomy: past, present, and future. Surg Technol Int 15:81–85PubMed
5.
go back to reference Harrell AG, Heniford BT (2005) Minimally invasive abdominal surgery: lux et veritas past, present, and future. Am J Surg 190:239–243CrossRefPubMed Harrell AG, Heniford BT (2005) Minimally invasive abdominal surgery: lux et veritas past, present, and future. Am J Surg 190:239–243CrossRefPubMed
6.
go back to reference Fuchs H, Livingston MA, Raskar R, Colucci D, Keller K, State A, Crawford JR, Rademacher P, Drake SH, Meyer AA (1998) Augmented reality visualization for laparoscopic surgery. In: Proceedings of the first international conference on medical image computing and computer-assisted intervention. Lecture Notes in Computer Science, vol 1496. Springer Berlin/Heidelberg Cambridge, pp 934–943 Fuchs H, Livingston MA, Raskar R, Colucci D, Keller K, State A, Crawford JR, Rademacher P, Drake SH, Meyer AA (1998) Augmented reality visualization for laparoscopic surgery. In: Proceedings of the first international conference on medical image computing and computer-assisted intervention. Lecture Notes in Computer Science, vol 1496. Springer Berlin/Heidelberg Cambridge, pp 934–943
7.
go back to reference Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented reality-assisted laparoscopic adrenalectomy. JAMA 292:2214–2215CrossRefPubMed Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented reality-assisted laparoscopic adrenalectomy. JAMA 292:2214–2215CrossRefPubMed
8.
go back to reference Mutter D, Bouras G, Marescaux J (2005) Digital technologies and quality improvement in cancer surgery. Eur J Surg Oncol 31:689–694CrossRefPubMed Mutter D, Bouras G, Marescaux J (2005) Digital technologies and quality improvement in cancer surgery. Eur J Surg Oncol 31:689–694CrossRefPubMed
9.
go back to reference Walimbe V, Shekhar R (2006) Automatic elastic image registration by interpolation of 3D rotations and translations from discrete rigid-body transformations. Med Image Anal 10:899–914CrossRefPubMed Walimbe V, Shekhar R (2006) Automatic elastic image registration by interpolation of 3D rotations and translations from discrete rigid-body transformations. Med Image Anal 10:899–914CrossRefPubMed
10.
go back to reference Dandekar O, Shekhar R (2007) FPGA-accelerated deformable image registration for improved target-delineation during CT-guided interventions. IEEE Trans Biomed Circuits Syst 1:116–127CrossRef Dandekar O, Shekhar R (2007) FPGA-accelerated deformable image registration for improved target-delineation during CT-guided interventions. IEEE Trans Biomed Circuits Syst 1:116–127CrossRef
11.
go back to reference Dandekar O, Castro-Pareja C, Shekhar R (2007) FPGA-based real-time 3D image preprocessing for image-guided medical interventions. J Real-Time Image Proc 1:285–301CrossRef Dandekar O, Castro-Pareja C, Shekhar R (2007) FPGA-based real-time 3D image preprocessing for image-guided medical interventions. J Real-Time Image Proc 1:285–301CrossRef
12.
go back to reference Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27:355–369CrossRefPubMed Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27:355–369CrossRefPubMed
13.
go back to reference Shahidi R, Bax MR, Maurer CR Jr, Johnson JA, Wilkinson EP, Wang B, West JB, Citardi MJ, Manwaring KH, Khadem R (2002) Implementation, calibration, and accuracy testing of an image-enhanced endoscopy system. IEEE Trans Med Imaging 21:1524–1535CrossRefPubMed Shahidi R, Bax MR, Maurer CR Jr, Johnson JA, Wilkinson EP, Wang B, West JB, Citardi MJ, Manwaring KH, Khadem R (2002) Implementation, calibration, and accuracy testing of an image-enhanced endoscopy system. IEEE Trans Med Imaging 21:1524–1535CrossRefPubMed
15.
go back to reference Hagiike M, Phillips EH, Berci G (2007) Performance differences in laparoscopic surgical skills between true high-definition and three-chip CCD video systems. Surg Endosc 21:1849–1854CrossRefPubMed Hagiike M, Phillips EH, Berci G (2007) Performance differences in laparoscopic surgical skills between true high-definition and three-chip CCD video systems. Surg Endosc 21:1849–1854CrossRefPubMed
16.
go back to reference Pierre SA, Ferrandino MN, Simmons WN, Fernandez C, Zhong P, Albala DM, Preminger GM (2009) High-definition laparoscopy: objective assessment of performance characteristics and comparison with standard laparoscopy. J Endourol 23:523–528CrossRefPubMed Pierre SA, Ferrandino MN, Simmons WN, Fernandez C, Zhong P, Albala DM, Preminger GM (2009) High-definition laparoscopy: objective assessment of performance characteristics and comparison with standard laparoscopy. J Endourol 23:523–528CrossRefPubMed
17.
go back to reference Miller A, Allen P, Fowler D (2004) In vivo stereoscopic imaging system with 5 degrees of freedom for minimal access surgery. Stud Health Technol Inform 98:234–240PubMed Miller A, Allen P, Fowler D (2004) In vivo stereoscopic imaging system with 5 degrees of freedom for minimal access surgery. Stud Health Technol Inform 98:234–240PubMed
18.
go back to reference Cannon JW, Stoll JA, Salgo IS, Knowles HB, Howe RD, Dupont PE, Marx GR, del Nido PJ (2003) Real-time three-dimensional ultrasound for guiding surgical tasks. Comput Aided Surg 8:82–90CrossRefPubMed Cannon JW, Stoll JA, Salgo IS, Knowles HB, Howe RD, Dupont PE, Marx GR, del Nido PJ (2003) Real-time three-dimensional ultrasound for guiding surgical tasks. Comput Aided Surg 8:82–90CrossRefPubMed
19.
go back to reference Sugeng L, Weinert L, Thiele K, Lang RM (2003) Real-time three-dimensional echocardiography using a novel matrix array transducer. Echocardiography 20:623–635CrossRefPubMed Sugeng L, Weinert L, Thiele K, Lang RM (2003) Real-time three-dimensional echocardiography using a novel matrix array transducer. Echocardiography 20:623–635CrossRefPubMed
21.
go back to reference Rogalla P, Kloeters C, Hein PA (2009) CT technology overview: 64-slice and beyond. Radiol Clin North Am 47:1–11CrossRefPubMed Rogalla P, Kloeters C, Hein PA (2009) CT technology overview: 64-slice and beyond. Radiol Clin North Am 47:1–11CrossRefPubMed
22.
go back to reference Dandekar O, Siddiqui J, Walimbe V, Shekhar R (2006) Image registration accuracy with low-dose CT: how low can we go? In: Proceedings of the third IEEE international symposium on biomedical imaging: nano to macro, Institute of Electrical and Electronics Engineers Computer Society, Arlington, pp 502–505 Dandekar O, Siddiqui J, Walimbe V, Shekhar R (2006) Image registration accuracy with low-dose CT: how low can we go? In: Proceedings of the third IEEE international symposium on biomedical imaging: nano to macro, Institute of Electrical and Electronics Engineers Computer Society, Arlington, pp 502–505
23.
go back to reference Crane NJ, McHone B, Hawksworth J, Pearl JP, Denobile J, Tadaki D, Pinto PA, Levin IW, Elster EA (2008) Enhanced surgical imaging: laparoscopic vessel identification and assessment of tissue oxygenation. J Am Coll Surg 206:1159–1166CrossRefPubMed Crane NJ, McHone B, Hawksworth J, Pearl JP, Denobile J, Tadaki D, Pinto PA, Levin IW, Elster EA (2008) Enhanced surgical imaging: laparoscopic vessel identification and assessment of tissue oxygenation. J Am Coll Surg 206:1159–1166CrossRefPubMed
Metadata
Title
Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography
Authors
Raj Shekhar
Omkar Dandekar
Venkatesh Bhat
Mathew Philip
Peng Lei
Carlos Godinez
Erica Sutton
Ivan George
Steven Kavic
Reuben Mezrich
Adrian Park
Publication date
01-08-2010
Publisher
Springer-Verlag
Published in
Surgical Endoscopy / Issue 8/2010
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-010-0890-8

Other articles of this Issue 8/2010

Surgical Endoscopy 8/2010 Go to the issue