Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 1/2020

Open Access 01-01-2020 | Mastoidectomy | Original Article

CIGuide: in situ augmented reality laser guidance

Authors: Zoltán Bárdosi, Christian Plattner, Yusuf Özbek, Thomas Hofmann, Srdjan Milosavljevic, Volker Schartinger, Wolfgang Freysinger

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 1/2020

Login to get access

Abstract

Purpose 

A robotic intraoperative laser guidance system with hybrid optic-magnetic tracking for skull base surgery is presented. It provides in situ augmented reality guidance for microscopic interventions at the lateral skull base with minimal mental and workload overhead on surgeons working without a monitor and dedicated pointing tools.

Methods 

Three components were developed: a registration tool (Rhinospider), a hybrid magneto-optic-tracked robotic feedback control scheme and a modified robotic end-effector. Rhinospider optimizes registration of patient and preoperative CT data by excluding user errors in fiducial localization with magnetic tracking. The hybrid controller uses an integrated microscope HD camera for robotic control with a guidance beam shining on a dual plate setup avoiding magnetic field distortions. A robotic needle insertion platform (iSYS Medizintechnik GmbH, Austria) was modified to position a laser beam with high precision in a surgical scene compatible to microscopic surgery.

Results 

System accuracy was evaluated quantitatively at various target positions on a phantom. The accuracy found is 1.2 mm ± 0.5 mm. Errors are primarily due to magnetic tracking. This application accuracy seems suitable for most surgical procedures in the lateral skull base. The system was evaluated quantitatively during a mastoidectomy of an anatomic head specimen and was judged useful by the surgeon.

Conclusion 

A hybrid robotic laser guidance system with direct visual feedback is proposed for navigated drilling and intraoperative structure localization. The system provides visual cues directly on/in the patient anatomy, reducing the standard limitations of AR visualizations like depth perception. The custom- built end-effector for the iSYS robot is transparent to using surgical microscopes and compatible with magnetic tracking. The cadaver experiment showed that guidance was accurate and that the end-effector is unobtrusive. This laser guidance has potential to aid the surgeon in finding the optimal mastoidectomy trajectory in more difficult interventions.
Literature
1.
go back to reference Edwards PJ, King AP, Maurer CR Jr, de Cunha DA, Hawkes DJ, Gaston RP, Fenlon MR, Jusczyzck A, Strong AJ, Chandler CL, Gleeson MJ (2000) Design and evaluation of a system for microscope-assisted guided interventions (MAGI). IEEE Trans Med Imaging 19(11):1082–1093CrossRef Edwards PJ, King AP, Maurer CR Jr, de Cunha DA, Hawkes DJ, Gaston RP, Fenlon MR, Jusczyzck A, Strong AJ, Chandler CL, Gleeson MJ (2000) Design and evaluation of a system for microscope-assisted guided interventions (MAGI). IEEE Trans Med Imaging 19(11):1082–1093CrossRef
2.
go back to reference Caversaccio MD, Freysinger W (2003) Computer-assistance for intraoperative navigation in ENT surgery. Minim Invasive Ther Allied Technol 12(1–2):36–51CrossRef Caversaccio MD, Freysinger W (2003) Computer-assistance for intraoperative navigation in ENT surgery. Minim Invasive Ther Allied Technol 12(1–2):36–51CrossRef
3.
go back to reference Kral F, Puschban EJ, Riechelmann H, Freysinger W (2013) Comparison of optical and electromagnetic tracking for navigated lateral skull base surgery. Int J Med Robot Comput Assist Surg 9(2):247–252CrossRef Kral F, Puschban EJ, Riechelmann H, Freysinger W (2013) Comparison of optical and electromagnetic tracking for navigated lateral skull base surgery. Int J Med Robot Comput Assist Surg 9(2):247–252CrossRef
4.
go back to reference Goebbels G, Troche K, Braun M, Ivanovic A, Grab A, von Lübtow K, Sader R, Zeilhofer F, Albrecht K, Praxmarer K (2003) ARSyS-Tricorder - Entwicklung eines Augmented Reality Systems für die intraoperative Navigation in der MKG Chirurgie. Fahlbusch, R.; Deutsche Gesellschaft für Computer- und Roboterassistierte Chirurgie: Zweite Jahrestagung der Deutschen Gesellschaft für Computer- und Roboterassistierte Chirurgie, CURAC. Germany Erlangen-Nürnberg, Nürnberg. http://publica.fraunhofer.de/documents/N-54460.html Goebbels G, Troche K, Braun M, Ivanovic A, Grab A, von Lübtow K, Sader R, Zeilhofer F, Albrecht K, Praxmarer K (2003) ARSyS-Tricorder - Entwicklung eines Augmented Reality Systems für die intraoperative Navigation in der MKG Chirurgie. Fahlbusch, R.; Deutsche Gesellschaft für Computer- und Roboterassistierte Chirurgie: Zweite Jahrestagung der Deutschen Gesellschaft für Computer- und Roboterassistierte Chirurgie, CURAC. Germany Erlangen-Nürnberg, Nürnberg. http://​publica.​fraunhofer.​de/​documents/​N-54460.​html
5.
go back to reference Hoppe (2004) Projektorbasierte erweiterte Realität in der rechnergestützten Chirurgie. Ph.D. thesis, Karlsruhe University Hoppe (2004) Projektorbasierte erweiterte Realität in der rechnergestützten Chirurgie. Ph.D. thesis, Karlsruhe University
6.
go back to reference Léger É, Drouin S, Louis Colins D, Popa T, Kersten-Oertel M (2017) Quantifying attention shifts in augmented reality image-guided neurosurgery. Healthc Technol Lett 4(5):188–192CrossRef Léger É, Drouin S, Louis Colins D, Popa T, Kersten-Oertel M (2017) Quantifying attention shifts in augmented reality image-guided neurosurgery. Healthc Technol Lett 4(5):188–192CrossRef
7.
go back to reference Meola A, Cutolo F, Carbone M, Cagnazzo F, Ferrari M, Ferrari V (2017) Augmented reality in neurosurgery: a systematic review. Neurosurg Rev 40(4):537–548CrossRef Meola A, Cutolo F, Carbone M, Cagnazzo F, Ferrari M, Ferrari V (2017) Augmented reality in neurosurgery: a systematic review. Neurosurg Rev 40(4):537–548CrossRef
8.
go back to reference Kytö M, Mäkinen A, Tossavainen T, Oittinen PT (2014) Stereoscopic depth perception in video see-through augmented reality within action space. J Electron Imaging 23(1):011006-1–011006-10CrossRef Kytö M, Mäkinen A, Tossavainen T, Oittinen PT (2014) Stereoscopic depth perception in video see-through augmented reality within action space. J Electron Imaging 23(1):011006-1–011006-10CrossRef
9.
go back to reference Mobasheri MH, Johnston M, Syed UM, King D, Darzi A (2015) The uses of smartphones and tablet devices in surgery: a systematic review of the literature. Surgery 158(5):1352–1371CrossRef Mobasheri MH, Johnston M, Syed UM, King D, Darzi A (2015) The uses of smartphones and tablet devices in surgery: a systematic review of the literature. Surgery 158(5):1352–1371CrossRef
10.
go back to reference Cutolo F, Meola A, Carbone M, Sinceri S, Cagnazzo F, Denaro E, Esposito N, Ferrari M, Ferrari V (2017) A new head-mounted display-based augmented reality system in neurosurgical oncology: a study on phantom. Comput Assist Surg 22(1):39–53CrossRef Cutolo F, Meola A, Carbone M, Sinceri S, Cagnazzo F, Denaro E, Esposito N, Ferrari M, Ferrari V (2017) A new head-mounted display-based augmented reality system in neurosurgical oncology: a study on phantom. Comput Assist Surg 22(1):39–53CrossRef
11.
go back to reference Cabrilo I, Bijlenga P, Schaller K (2014) Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir 156(9):1769–1774CrossRef Cabrilo I, Bijlenga P, Schaller K (2014) Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir 156(9):1769–1774CrossRef
12.
go back to reference Gavaghan K, Oliveira-Santos T, Peterhans M, Reyes M, Kim H, Anderegg S, Weber S (2012) Evaluation of a portable image overlay projector for the visualization of surgical navigation data: phantom studies. Int J CARS 7:547–556CrossRef Gavaghan K, Oliveira-Santos T, Peterhans M, Reyes M, Kim H, Anderegg S, Weber S (2012) Evaluation of a portable image overlay projector for the visualization of surgical navigation data: phantom studies. Int J CARS 7:547–556CrossRef
13.
go back to reference Tabrizi LB, Mahwash M (2015) Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123:206–211CrossRef Tabrizi LB, Mahwash M (2015) Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123:206–211CrossRef
14.
go back to reference Zeng B, Meng F, Ding H, Wang G (2017) A surgical robot with augmented reality visualization for stereoelectroencephalography electrode implantation. Int J CARS 12:1355–1368CrossRef Zeng B, Meng F, Ding H, Wang G (2017) A surgical robot with augmented reality visualization for stereoelectroencephalography electrode implantation. Int J CARS 12:1355–1368CrossRef
15.
go back to reference Herrlich M, Tavakol P, Black D, Wenig D, Rieder C, Malaka R, Kikinis R (2017) Instrument-mounted displays for reducing cognitive load during surgical navigation. Int J CARS 12:1599–1605CrossRef Herrlich M, Tavakol P, Black D, Wenig D, Rieder C, Malaka R, Kikinis R (2017) Instrument-mounted displays for reducing cognitive load during surgical navigation. Int J CARS 12:1599–1605CrossRef
16.
go back to reference Bardosi Z, Plattner C, Özbek Y, Hofmann T, Milosavljevic S, Freysinger W (2017) CIGuide: a hybrid-tracked robotic laser guidance platform for intraoperative guidance. In: CARS 31st international congress and exhibition, Barcelona, June 20–24 Bardosi Z, Plattner C, Özbek Y, Hofmann T, Milosavljevic S, Freysinger W (2017) CIGuide: a hybrid-tracked robotic laser guidance platform for intraoperative guidance. In: CARS 31st international congress and exhibition, Barcelona, June 20–24
17.
go back to reference Bardosi Z, Plattner C, Özbek Y, Hofmann T, Milosavljevic S, Freysinger W (2017) CIGuide: an intraoperative hybrid-tracker robotic laser guidance platform. In: 61st Austrian ENT Congress, Wien, September 13–17 Bardosi Z, Plattner C, Özbek Y, Hofmann T, Milosavljevic S, Freysinger W (2017) CIGuide: an intraoperative hybrid-tracker robotic laser guidance platform. In: 61st Austrian ENT Congress, Wien, September 13–17
18.
go back to reference Bardosi Z (2018) Predicting accuracies in spatial augmented reality. Ph.D. Thesis, Innsbruck Bardosi Z (2018) Predicting accuracies in spatial augmented reality. Ph.D. Thesis, Innsbruck
19.
go back to reference Kettenbach J, Kara L, Toporek G, Fuerst M, Kronreif G (2014) A robotic needle-positioning and guidance system for CT-guided puncture: ex vivo results. Minim Invasive Ther Allied Technol 23:271–278CrossRef Kettenbach J, Kara L, Toporek G, Fuerst M, Kronreif G (2014) A robotic needle-positioning and guidance system for CT-guided puncture: ex vivo results. Minim Invasive Ther Allied Technol 23:271–278CrossRef
20.
go back to reference Gabloner T (2015) Closed loop controller design for robot-based needle orientation in radiofrequency ablation. MSc. Thesis, MCI, Mechatronics Mechanical Engineering, Innsbruck, Austria Gabloner T (2015) Closed loop controller design for robot-based needle orientation in radiofrequency ablation. MSc. Thesis, MCI, Mechatronics Mechanical Engineering, Innsbruck, Austria
21.
go back to reference Bardosi ZR, Özbek Y, Plattner C, Freysinger W (2013) Auf dem Weg zum Heiligen Gral der 3D-Navigation submillimetrische Anwendungsgenauigkeit im Felsenbein. In: W Freysinger (ed)., 12th annual conference of the German Society for computer- and roboter-assisted surgery (CURAC), pp 155–158 Bardosi ZR, Özbek Y, Plattner C, Freysinger W (2013) Auf dem Weg zum Heiligen Gral der 3D-Navigation submillimetrische Anwendungsgenauigkeit im Felsenbein. In: W Freysinger (ed)., 12th annual conference of the German Society for computer- and roboter-assisted surgery (CURAC), pp 155–158
22.
go back to reference Zhang Z (2016) Camera calibration: a personal retrospective. Mach Vis Appl 27(7):963–965CrossRef Zhang Z (2016) Camera calibration: a personal retrospective. Mach Vis Appl 27(7):963–965CrossRef
23.
go back to reference Gao XS, Hou XR, Tang J, Chang HF (2003) Complete solution classification for the perspective-three-point problem. IEEE Trans Pattern Anal Mach Intell 25(8):930–943CrossRef Gao XS, Hou XR, Tang J, Chang HF (2003) Complete solution classification for the perspective-three-point problem. IEEE Trans Pattern Anal Mach Intell 25(8):930–943CrossRef
24.
go back to reference Lepetit V, Moreno-Noguer F, Fua P (2009) EPnP: an accurate O(n) solution to the Pnp problem. Int J Comput Vis 81:155–166CrossRef Lepetit V, Moreno-Noguer F, Fua P (2009) EPnP: an accurate O(n) solution to the Pnp problem. Int J Comput Vis 81:155–166CrossRef
25.
go back to reference Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin J-C, Pujol S, Bauer C, Jennings D, Fennessy FM, Sonka M, Buatti J, Aylward SR, Miller JV, Pieper S, Kikinis R (2012) 3D Slicer as an image computing platform for the quantitative imaging network. Magn Reson Imaging 30(9):1323–1341CrossRef Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin J-C, Pujol S, Bauer C, Jennings D, Fennessy FM, Sonka M, Buatti J, Aylward SR, Miller JV, Pieper S, Kikinis R (2012) 3D Slicer as an image computing platform for the quantitative imaging network. Magn Reson Imaging 30(9):1323–1341CrossRef
26.
go back to reference Ganglberger F, Özbek Y, Freysinger W (2013) The use of common toolkits CTK in the computer-assisted surgery—a demo application. In: Proceedings of the 12th annual conference of the German Society for computer—and robot-assisted surgery (CURAC), pp 161–164 Ganglberger F, Özbek Y, Freysinger W (2013) The use of common toolkits CTK in the computer-assisted surgery—a demo application. In: Proceedings of the 12th annual conference of the German Society for computer—and robot-assisted surgery (CURAC), pp 161–164
27.
go back to reference Schroeder W, Martin K, Lorensen B (2006) The visualization toolkit, 4th edn. Kitware. ISBN 978-1-930934-19-1 Schroeder W, Martin K, Lorensen B (2006) The visualization toolkit, 4th edn. Kitware. ISBN 978-1-930934-19-1
28.
go back to reference Yoo TS, Ackerman MJ, Lorensen WE, Schroeder W, Chalana V, Aylward S, Metaxas D, Whitaker R (2002) Engineering and algorithm design for an image processing API: a technical report on ITK—the insight toolkit. In: J Westwood (ed.), Proceedings medicine meets virtual reality, pp 586–592. IOS Press, Amsterdam Yoo TS, Ackerman MJ, Lorensen WE, Schroeder W, Chalana V, Aylward S, Metaxas D, Whitaker R (2002) Engineering and algorithm design for an image processing API: a technical report on ITK—the insight toolkit. In: J Westwood (ed.), Proceedings medicine meets virtual reality, pp 586–592. IOS Press, Amsterdam
29.
go back to reference Enquobahrie A, Cheng P, Gary K, Ibanez L, Gobbi D, Lindseth F, Yaniv Z, Aylward S, Jomier J, Cleary K (2007) The image-guided surgery toolkit IGSTK: an open source C++ software toolkit. J Digit Imaging 20(1):21–33CrossRef Enquobahrie A, Cheng P, Gary K, Ibanez L, Gobbi D, Lindseth F, Yaniv Z, Aylward S, Jomier J, Cleary K (2007) The image-guided surgery toolkit IGSTK: an open source C++ software toolkit. J Digit Imaging 20(1):21–33CrossRef
30.
go back to reference Tokuda J, Fischer GS, Papademetris X, Yaniv Z, Ibanez L, Cheng P, Liu H, Blevins J, Arata J, Golby AJ, Kapur T, Pieper S, Burdette EC, Fichtinger G, Tempany CM, Hata N (2009) OpenIGTLink: an open network protocol for image-guided therapy environment. Int J Med Robot 5(4):423–434CrossRef Tokuda J, Fischer GS, Papademetris X, Yaniv Z, Ibanez L, Cheng P, Liu H, Blevins J, Arata J, Golby AJ, Kapur T, Pieper S, Burdette EC, Fichtinger G, Tempany CM, Hata N (2009) OpenIGTLink: an open network protocol for image-guided therapy environment. Int J Med Robot 5(4):423–434CrossRef
31.
go back to reference Bradski G (2000) The openCV library. Dr. Dobb’s J Softw Tools 25:120–125 Bradski G (2000) The openCV library. Dr. Dobb’s J Softw Tools 25:120–125
32.
go back to reference Horn Berthold KP (1987) Closed-form solution of absolute orientation using unit quaternions. J Opt Soc Am 4:629–642CrossRef Horn Berthold KP (1987) Closed-form solution of absolute orientation using unit quaternions. J Opt Soc Am 4:629–642CrossRef
33.
go back to reference Fitzpatrick JM, West JB (2001) The distribution of target registration error in rigid-body point-based registration. IEEE Trans Med Imaging 20(9):917–927CrossRef Fitzpatrick JM, West JB (2001) The distribution of target registration error in rigid-body point-based registration. IEEE Trans Med Imaging 20(9):917–927CrossRef
34.
go back to reference Hofmann T (2016) Fusing magnetic and optical sensor data for optimized axis alignment of medicine robot. MSc. Thesis, MCI, Innsbruck, Austria Hofmann T (2016) Fusing magnetic and optical sensor data for optimized axis alignment of medicine robot. MSc. Thesis, MCI, Innsbruck, Austria
Metadata
Title
CIGuide: in situ augmented reality laser guidance
Authors
Zoltán Bárdosi
Christian Plattner
Yusuf Özbek
Thomas Hofmann
Srdjan Milosavljevic
Volker Schartinger
Wolfgang Freysinger
Publication date
01-01-2020
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 1/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-019-02066-1

Other articles of this Issue 1/2020

International Journal of Computer Assisted Radiology and Surgery 1/2020 Go to the issue