Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 6/2016

01-06-2016 | Original Article

On-demand calibration and evaluation for electromagnetically tracked laparoscope in augmented reality visualization

Authors: Xinyang Liu, William Plishker, George Zaki, Sukryool Kang, Timothy D. Kane, Raj Shekhar

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 6/2016

Login to get access

Abstract

Purpose

Common camera calibration methods employed in current laparoscopic augmented reality systems require the acquisition of multiple images of an entire checkerboard pattern from various poses. This lengthy procedure prevents performing laparoscope calibration in the operating room (OR). The purpose of this work was to develop a fast calibration method for electromagnetically (EM) tracked laparoscopes, such that the calibration can be performed in the OR on demand.

Methods

We designed a mechanical tracking mount to uniquely and snugly position an EM sensor to an appropriate location on a conventional laparoscope. A tool named fCalib was developed to calibrate intrinsic camera parameters, distortion coefficients, and extrinsic parameters (transformation between the scope lens coordinate system and the EM sensor coordinate system) using a single image that shows an arbitrary portion of a special target pattern. For quick evaluation of calibration results in the OR, we integrated a tube phantom with fCalib prototype and overlaid a virtual representation of the tube on the live video scene.

Results

We compared spatial target registration error between the common OpenCV method and the fCalib method in a laboratory setting. In addition, we compared the calibration re-projection error between the EM tracking-based fCalib and the optical tracking-based fCalib in a clinical setting. Our results suggest that the proposed method is comparable to the OpenCV method. However, changing the environment, e.g., inserting or removing surgical tools, might affect re-projection accuracy for the EM tracking-based approach. Computational time of the fCalib method averaged 14.0 s (range 3.5 s–22.7 s).

Conclusions

We developed and validated a prototype for fast calibration and evaluation of EM tracked conventional (forward viewing) laparoscopes. The calibration method achieved acceptable accuracy and was relatively fast and easy to be performed in the OR on demand.
Appendix
Available only for authorised users
Literature
1.
go back to reference Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27(3):355–369CrossRefPubMed Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27(3):355–369CrossRefPubMed
2.
go back to reference Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C, Sutton E, George I, Kavic S, Mezrich R, Park A (2010) Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24(8):1976–1985CrossRefPubMed Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C, Sutton E, George I, Kavic S, Mezrich R, Park A (2010) Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24(8):1976–1985CrossRefPubMed
3.
go back to reference Leven J, Burschka D, Kumar R, Zhang G, Blumenkranz S, Dai XD, Awad M, Hager GD, Marohn M, Choti M, Hasser C, Taylor RH (2005) DaVinci canvas: a telerobotic surgical system with integrated, robot-assisted, laparoscopic ultrasound capability. Proc Med Image Comput Comput Assist Interv 8(Pt 1):811–818 Leven J, Burschka D, Kumar R, Zhang G, Blumenkranz S, Dai XD, Awad M, Hager GD, Marohn M, Choti M, Hasser C, Taylor RH (2005) DaVinci canvas: a telerobotic surgical system with integrated, robot-assisted, laparoscopic ultrasound capability. Proc Med Image Comput Comput Assist Interv 8(Pt 1):811–818
4.
go back to reference Cheung CL, Wedlake C, Moore J, Pautler SE, Peters TM (2010) Fused video and ultrasound images for minimally invasive partial nephrectomy: a phantom study. Proc Med Image Comput Comput Assist Interv 13(Pt 3):408–415 Cheung CL, Wedlake C, Moore J, Pautler SE, Peters TM (2010) Fused video and ultrasound images for minimally invasive partial nephrectomy: a phantom study. Proc Med Image Comput Comput Assist Interv 13(Pt 3):408–415
5.
go back to reference Kang X, Azizian M, Wilson E, Wu K, Martin AD, Kane TD, Peters CA, Cleary K, Shekhar R (2014) Stereoscopic augmented reality for laparoscopic surgery. Surg Endosc 28(7):2227–2235CrossRefPubMed Kang X, Azizian M, Wilson E, Wu K, Martin AD, Kane TD, Peters CA, Cleary K, Shekhar R (2014) Stereoscopic augmented reality for laparoscopic surgery. Surg Endosc 28(7):2227–2235CrossRefPubMed
6.
go back to reference Shiu Y, Ahmad S (1989) Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form ax = xb. IEEE Trans Robot Autom 5(1):16–29CrossRef Shiu Y, Ahmad S (1989) Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form ax = xb. IEEE Trans Robot Autom 5(1):16–29CrossRef
8.
go back to reference Heikkila J, Silven O (1997) A four-step camera calibration procedure with implicit image correction. In: Proceedings of IEEE computer society conference computer vision pattern recognition. pp 1106–1112 Heikkila J, Silven O (1997) A four-step camera calibration procedure with implicit image correction. In: Proceedings of IEEE computer society conference computer vision pattern recognition. pp 1106–1112
9.
go back to reference Zhang Z (1999) Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of international conference on computer vision. pp 666–673 Zhang Z (1999) Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of international conference on computer vision. pp 666–673
10.
go back to reference Shekhar R, Liu X, Wilson E, Kang S, Petrosyan M, Kane TD (2015) Stereoscopic augmented reality visualization for laparoscopic surgery—initial clinical experience. In: Proceedings of annual meeting of society of American gastrointestinal and endoscopic surgeons Shekhar R, Liu X, Wilson E, Kang S, Petrosyan M, Kane TD (2015) Stereoscopic augmented reality visualization for laparoscopic surgery—initial clinical experience. In: Proceedings of annual meeting of society of American gastrointestinal and endoscopic surgeons
11.
go back to reference Barreto JP, Roquette J, Sturm P, Fonseca F (2009) Automatic camera calibration applied to medical endoscopy. In: Proceedings of british machine vision conference Barreto JP, Roquette J, Sturm P, Fonseca F (2009) Automatic camera calibration applied to medical endoscopy. In: Proceedings of british machine vision conference
12.
go back to reference Melo R, Barreto JP, Falcão G (2012) A new solution for camera calibration and real-time image distortion correction in medical endoscopy-initial technical evaluation. IEEE Trans Biomed Eng 59(3):634–644CrossRefPubMed Melo R, Barreto JP, Falcão G (2012) A new solution for camera calibration and real-time image distortion correction in medical endoscopy-initial technical evaluation. IEEE Trans Biomed Eng 59(3):634–644CrossRefPubMed
13.
go back to reference Liu X, Su H, Kang S, Kane TD, Shekhar R (2015) Application of single-image camera calibration for ultrasound augmented laparoscopic visualization. In: Proceedings of SPIE medical, imaging. p 94151T Liu X, Su H, Kang S, Kane TD, Shekhar R (2015) Application of single-image camera calibration for ultrasound augmented laparoscopic visualization. In: Proceedings of SPIE medical, imaging. p 94151T
14.
go back to reference Feuerstein M, Reichl T, Vogel J, Traub J, Navab N (2009) Magneto-optical tracking of flexible laparoscopic ultrasound: model-based online detection and correction of magnetic tracking errors. IEEE Trans Med Imaging 28(6):951–967CrossRefPubMed Feuerstein M, Reichl T, Vogel J, Traub J, Navab N (2009) Magneto-optical tracking of flexible laparoscopic ultrasound: model-based online detection and correction of magnetic tracking errors. IEEE Trans Med Imaging 28(6):951–967CrossRefPubMed
16.
go back to reference Franz AM, März K, Hummel J, Birkfellner W, Bendl R, Delorme S, Schlemmer HP, Meinzer HP, Maier-Hein L (2012) Electromagnetic tracking for US-guided interventions: standardized assessment of a new compact field generator. Int J Comput Assist Radiol Surg 7(6):813–818CrossRefPubMed Franz AM, März K, Hummel J, Birkfellner W, Bendl R, Delorme S, Schlemmer HP, Meinzer HP, Maier-Hein L (2012) Electromagnetic tracking for US-guided interventions: standardized assessment of a new compact field generator. Int J Comput Assist Radiol Surg 7(6):813–818CrossRefPubMed
17.
go back to reference Maier-Hein L, Franz AM, Birkfellner W, Hummel J, Gergel I, Wegner I, Meinzer HP (2012) Standardized assessment of new electromagnetic field generators in an interventional radiology setting. Med Phys 39(6):3424–3434CrossRefPubMed Maier-Hein L, Franz AM, Birkfellner W, Hummel J, Gergel I, Wegner I, Meinzer HP (2012) Standardized assessment of new electromagnetic field generators in an interventional radiology setting. Med Phys 39(6):3424–3434CrossRefPubMed
18.
go back to reference Moore JT, Wiles AD, Wedlake C, Bainbridge D, Kiaii B, Luisa Trejos A, Patel R, Peters TM (2010) Integration of trans-esophageal echocardiography with magnetic tracking technology for cardiac interventions. In: Proceedings of SPIE medical, imaging. p 76252Y Moore JT, Wiles AD, Wedlake C, Bainbridge D, Kiaii B, Luisa Trejos A, Patel R, Peters TM (2010) Integration of trans-esophageal echocardiography with magnetic tracking technology for cardiac interventions. In: Proceedings of SPIE medical, imaging. p 76252Y
19.
go back to reference Liu X, Kang S, Wilson E, Peters CA, Kane TD, Shekhar R (2014) Evaluation of electromagnetic tracking for stereoscopic augmented reality laparoscopic visualization. Proc MICCAI Workshop Clin Image Based Proced Transl Res Med Imaging 8361:84–91 Liu X, Kang S, Wilson E, Peters CA, Kane TD, Shekhar R (2014) Evaluation of electromagnetic tracking for stereoscopic augmented reality laparoscopic visualization. Proc MICCAI Workshop Clin Image Based Proced Transl Res Med Imaging 8361:84–91
20.
go back to reference Atcheson B, Heide F, Heidrich W (2010) CALTag: High precision fiducial markers for camera calibration. In: 15th International workshop on vision, modeling and visualization. Siegen Atcheson B, Heide F, Heidrich W (2010) CALTag: High precision fiducial markers for camera calibration. In: 15th International workshop on vision, modeling and visualization. Siegen
21.
go back to reference Johnson MP (2003) Exploiting quaternions to support expressive interactive character motion. Dissertation, MIT Johnson MP (2003) Exploiting quaternions to support expressive interactive character motion. Dissertation, MIT
22.
go back to reference Hartley RI, Sturm P (1997) Triangulation. Comput Vis Image Underst 68(2):146–157CrossRef Hartley RI, Sturm P (1997) Triangulation. Comput Vis Image Underst 68(2):146–157CrossRef
Metadata
Title
On-demand calibration and evaluation for electromagnetically tracked laparoscope in augmented reality visualization
Authors
Xinyang Liu
William Plishker
George Zaki
Sukryool Kang
Timothy D. Kane
Raj Shekhar
Publication date
01-06-2016
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 6/2016
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-016-1406-3

Other articles of this Issue 6/2016

International Journal of Computer Assisted Radiology and Surgery 6/2016 Go to the issue