Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 8/2017

01-08-2017 | Original Article

A surgical robot with augmented reality visualization for stereoelectroencephalography electrode implantation

Authors: Bowei Zeng, Fanle Meng, Hui Ding, Guangzhi Wang

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 8/2017

Login to get access

Abstract

Purpose

Using existing stereoelectroencephalography (SEEG) electrode implantation surgical robot systems, it is difficult to intuitively validate registration accuracy and display the electrode entry points (EPs) and the anatomical structure around the electrode trajectories in the patient space to the surgeon. This paper proposes a prototype system that can realize video see-through augmented reality (VAR) and spatial augmented reality (SAR) for SEEG implantation. The system helps the surgeon quickly and intuitively confirm the registration accuracy, locate EPs and visualize the internal anatomical structure in the image space and patient space.

Methods

We designed and developed a projector-camera system (PCS) attached to the distal flange of a robot arm. First, system calibration is performed. Second, the PCS is used to obtain the point clouds of the surface of the patient’s head, which are utilized for patient-to-image registration. Finally, VAR is produced by merging the real-time video of the patient and the preoperative three-dimensional (3D) operational planning model. In addition, SAR is implemented by projecting the planning electrode trajectories and local anatomical structure onto the patient’s scalp.

Results

The error of registration, the electrode EPs and the target points are evaluated on a phantom. The fiducial registration error is \(0.25 \pm 0.23\) mm (max 1.22 mm), and the target registration error is \(0.62\pm 0.28\) mm (max 1.18 mm). The projection overlay error is \(0.75\pm 0.52\) mm, and the TP error after the pre-warped projection is \(0.82\pm 0.23\) mm. The TP error caused by a surgeon’s viewpoint deviation is also evaluated.

Conclusion

The presented system can help surgeons quickly verify registration accuracy during SEEG procedures and can provide accurate EP locations and internal structural information to the surgeon. With more intuitive surgical information, the surgeon may have more confidence and be able to perform surgeries with better outcomes.
Literature
1.
go back to reference Cardinale F, Cossu M, Castana L, Casaceli G, Schiariti MP, Miserocchi A, Fuschillo D, Moscato A, Caborni C, Arnulfo G (2013) Stereoelectroencephalography: surgical methodology, safety, and stereotactic application accuracy in 500 procedures. Neurosurgery 72(3):353–366CrossRefPubMed Cardinale F, Cossu M, Castana L, Casaceli G, Schiariti MP, Miserocchi A, Fuschillo D, Moscato A, Caborni C, Arnulfo G (2013) Stereoelectroencephalography: surgical methodology, safety, and stereotactic application accuracy in 500 procedures. Neurosurgery 72(3):353–366CrossRefPubMed
2.
go back to reference Cossu M, Fuschillo D, Casaceli G, Pelliccia V, Castana L, Mai R, Francione S, Sartori I, Gozzo F, Nobili L (2015) Stereoelectroencephalography-guided radiofrequency thermocoagulation in the epileptogenic zone: a retrospective study on 89 cases. J Neurosurg 123(6):1358–1367CrossRefPubMed Cossu M, Fuschillo D, Casaceli G, Pelliccia V, Castana L, Mai R, Francione S, Sartori I, Gozzo F, Nobili L (2015) Stereoelectroencephalography-guided radiofrequency thermocoagulation in the epileptogenic zone: a retrospective study on 89 cases. J Neurosurg 123(6):1358–1367CrossRefPubMed
3.
go back to reference Nowell M, Rodionov R, Diehl B, Wehner T, Zombori G, Kinghorn J, Ourselin S, Duncan J, Miserocchi A, McEvoy A (2014) A novel method for implementation of frameless StereoEEG in epilepsy surgery. Neurosurgery 10(4):525CrossRefPubMedPubMedCentral Nowell M, Rodionov R, Diehl B, Wehner T, Zombori G, Kinghorn J, Ourselin S, Duncan J, Miserocchi A, McEvoy A (2014) A novel method for implementation of frameless StereoEEG in epilepsy surgery. Neurosurgery 10(4):525CrossRefPubMedPubMedCentral
4.
go back to reference Faria C, Erlhagen W, Rito M, De Momi E, Ferrigno G, Bicho E (2015) Review of robotic technology for stereotactic neurosurgery. IEEE Rev Biomed Eng 8:125–137CrossRefPubMed Faria C, Erlhagen W, Rito M, De Momi E, Ferrigno G, Bicho E (2015) Review of robotic technology for stereotactic neurosurgery. IEEE Rev Biomed Eng 8:125–137CrossRefPubMed
6.
go back to reference Besharati Tabrizi L, Mahvash M (2015) Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123(1):206–211CrossRefPubMed Besharati Tabrizi L, Mahvash M (2015) Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123(1):206–211CrossRefPubMed
8.
go back to reference Kersten-Oertel M, Gerard I, Drouin S, Mok K, Sirhan D, Sinclair DS, Collins DL (2015) Augmented reality in neurovascular surgery: feasibility and first uses in the operating room. Int J Comput Assist Radiol Surg 10(11):1823–1836CrossRefPubMed Kersten-Oertel M, Gerard I, Drouin S, Mok K, Sirhan D, Sinclair DS, Collins DL (2015) Augmented reality in neurovascular surgery: feasibility and first uses in the operating room. Int J Comput Assist Radiol Surg 10(11):1823–1836CrossRefPubMed
9.
go back to reference González-Martínez J, Bulacio J, Thompson S, Gale J, Smithason S, Najm I, Bingaman W (2016) Technique, results, and complications related to robot-assisted stereoelectroencephalography. Neurosurgery 78(2):169–180CrossRefPubMed González-Martínez J, Bulacio J, Thompson S, Gale J, Smithason S, Najm I, Bingaman W (2016) Technique, results, and complications related to robot-assisted stereoelectroencephalography. Neurosurgery 78(2):169–180CrossRefPubMed
10.
go back to reference Watanabe E, Satoh M, Konno T, Hirai M, Yamaguchi T (2016) The trans-visible navigator: a see-through neuronavigation system using augmented reality. World Neurosurg 87:399–405CrossRefPubMed Watanabe E, Satoh M, Konno T, Hirai M, Yamaguchi T (2016) The trans-visible navigator: a see-through neuronavigation system using augmented reality. World Neurosurg 87:399–405CrossRefPubMed
11.
go back to reference Kockro RA, Tsai YT, Ng I, Hwang P, Zhu C, Agusanto K, Hong LX, Serra L (2009) DEX - RAY: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery 65(4):795–808CrossRefPubMed Kockro RA, Tsai YT, Ng I, Hwang P, Zhu C, Agusanto K, Hong LX, Serra L (2009) DEX - RAY: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery 65(4):795–808CrossRefPubMed
12.
go back to reference Park H, Kang G-C, Lee M-H, Kim S-J, Park J-I (2005) Direct-projected augmented reality considering user’s viewpoint. In: Proceedings of international meeting on information display and exhibition, pp 748–751 Park H, Kang G-C, Lee M-H, Kim S-J, Park J-I (2005) Direct-projected augmented reality considering user’s viewpoint. In: Proceedings of international meeting on information display and exhibition, pp 748–751
13.
go back to reference Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201CrossRefPubMed Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201CrossRefPubMed
14.
go back to reference Hennersperger C, Manus J, Navab N (2016) Mobile laserprojection in computer assisted neurosurgery. In: International conference on medical imaging and virtual reality. Springer, pp 151–162 Hennersperger C, Manus J, Navab N (2016) Mobile laserprojection in computer assisted neurosurgery. In: International conference on medical imaging and virtual reality. Springer, pp 151–162
15.
go back to reference Geng J (2011) Structured-light 3D surface imaging: a tutorial. Adv Opt Phot 3(2):128–160CrossRef Geng J (2011) Structured-light 3D surface imaging: a tutorial. Adv Opt Phot 3(2):128–160CrossRef
16.
go back to reference Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge
17.
go back to reference Moreno D, Taubin G (2012) Simple, accurate, and robust projector-camera calibration. In: 2012 Second international conference on 3D imaging, modeling, processing, visualization & transmission. IEEE, pp 464–471 Moreno D, Taubin G (2012) Simple, accurate, and robust projector-camera calibration. In: 2012 Second international conference on 3D imaging, modeling, processing, visualization & transmission. IEEE, pp 464–471
18.
go back to reference Tsai RY, Lenz RK (1989) A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans on Robot Autom 5(3):345–358CrossRef Tsai RY, Lenz RK (1989) A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans on Robot Autom 5(3):345–358CrossRef
19.
go back to reference Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Patt Anal Mach Intell 22(11):1330–1334CrossRef Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Patt Anal Mach Intell 22(11):1330–1334CrossRef
20.
go back to reference Besl PJ, McKay ND (1992) Method for registration of 3-D shapes. In: Robotics-DL tentative. International society for optics and photonics, pp 586–606 Besl PJ, McKay ND (1992) Method for registration of 3-D shapes. In: Robotics-DL tentative. International society for optics and photonics, pp 586–606
21.
go back to reference Rusinkiewicz S, Levoy M (2001) Efficient variants of the ICP algorithm. In: 3-D Digital imaging and modeling. Proceedings. Third international conference on, 2001. IEEE, pp 145–152 Rusinkiewicz S, Levoy M (2001) Efficient variants of the ICP algorithm. In: 3-D Digital imaging and modeling. Proceedings. Third international conference on, 2001. IEEE, pp 145–152
22.
go back to reference Sielhorst T, Bichlmeier C, Heining SM, Navab N (2006) Depth perception—a major issue in medical AR: evaluation study by twenty surgeons. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 364–372 Sielhorst T, Bichlmeier C, Heining SM, Navab N (2006) Depth perception—a major issue in medical AR: evaluation study by twenty surgeons. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 364–372
23.
go back to reference Cheng O, Guangzhi W, Quan Z, Wei K, Hui D (2005) Evaluating harris method in camera calibration. In: Engineering in medicine and biology society. IEEE-EMBS 2005. 27th annual international conference of the. IEEE, pp 6383–6386 Cheng O, Guangzhi W, Quan Z, Wei K, Hui D (2005) Evaluating harris method in camera calibration. In: Engineering in medicine and biology society. IEEE-EMBS 2005. 27th annual international conference of the. IEEE, pp 6383–6386
24.
go back to reference Zhu C, Liang X, Kockro R, Serra L (2004) Accuracy evaluation of an augmented reality enhanced surgical navigation system. In: International congress series. Elsevier, p 1329 Zhu C, Liang X, Kockro R, Serra L (2004) Accuracy evaluation of an augmented reality enhanced surgical navigation system. In: International congress series. Elsevier, p 1329
25.
go back to reference Roessler K, Sommer B, Merkel A, Rampp S, Gollwitzer S, Hamer HM, Buchfelder M (2016) A frameless stereotactic implantation technique for depth electrodes in refractory epilepsy using intraoperative magnetic resonance imaging. World Neurosurg 94:206–210CrossRefPubMed Roessler K, Sommer B, Merkel A, Rampp S, Gollwitzer S, Hamer HM, Buchfelder M (2016) A frameless stereotactic implantation technique for depth electrodes in refractory epilepsy using intraoperative magnetic resonance imaging. World Neurosurg 94:206–210CrossRefPubMed
26.
go back to reference Vakharia VN, Sparks R, O’Keeffe AG, Rodionov R, Miserocchi A, McEvoy A, Ourselin S, Duncan J (2017) Accuracy of intracranial electrode placement for stereoelectroencephalography: A systematic review and meta-analysis. Epilepsia 58(6):921–932 Vakharia VN, Sparks R, O’Keeffe AG, Rodionov R, Miserocchi A, McEvoy A, Ourselin S, Duncan J (2017) Accuracy of intracranial electrode placement for stereoelectroencephalography: A systematic review and meta-analysis. Epilepsia 58(6):921–932
27.
go back to reference Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Munter MW, Debus J, Harms W (2008) Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. Int J Radiat Oncol BiolPhys 70(3):944–952CrossRef Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Munter MW, Debus J, Harms W (2008) Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. Int J Radiat Oncol BiolPhys 70(3):944–952CrossRef
Metadata
Title
A surgical robot with augmented reality visualization for stereoelectroencephalography electrode implantation
Authors
Bowei Zeng
Fanle Meng
Hui Ding
Guangzhi Wang
Publication date
01-08-2017
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 8/2017
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-017-1634-1

Other articles of this Issue 8/2017

International Journal of Computer Assisted Radiology and Surgery 8/2017 Go to the issue