Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 6/2019

01-06-2019 | Original Article

Preliminary study of an RNN-based active interventional robotic system (AIRS) in retinal microsurgery

Authors: Changyan He, Niravkumar Patel, Ali Ebrahimi, Marin Kobilarov, Iulian Iordachita

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 6/2019

Login to get access

Abstract

Purpose

Retinal microsurgery requires highly dexterous and precise maneuvering of instruments inserted into the eyeball through the sclerotomy port. During such procedures, the sclera can potentially be injured from extreme tool-to-sclera contact force caused by surgeon’s unintentional misoperations.

Methods

We present an active interventional robotic system to prevent such iatrogenic accidents by enabling the robotic system to actively counteract the surgeon’s possible unsafe operations in advance of their occurrence. Relying on a novel force sensing tool to measure and collect scleral forces, we construct a recurrent neural network with long short-term memory unit to oversee surgeon’s operation and predict possible unsafe scleral forces up to the next 200 ms. We then apply a linear admittance control to actuate the robot to reduce the undesired scleral force. The system is implemented using an existing “steady hand” eye robot platform. The proposed method is evaluated on an artificial eye phantom by performing a “vessel following” mock retinal surgery operation.

Results

Empirical validation over multiple trials indicates that the proposed active interventional robotic system could help to reduce the number of unsafe manipulation events.

Conclusions

We develop an active interventional robotic system to actively prevent surgeon’s unsafe operations in retinal surgery. The result of the evaluation experiments shows that the proposed system can improve the surgeon’s performance.
Literature
1.
go back to reference Rahimy E, Wilson J, Tsao T, Schwartz S, Hubschman J (2013) Robot-assisted intraocular surgery: development of the iriss and feasibility studies in an animal model. Eye 27(8):972CrossRefPubMedPubMedCentral Rahimy E, Wilson J, Tsao T, Schwartz S, Hubschman J (2013) Robot-assisted intraocular surgery: development of the iriss and feasibility studies in an animal model. Eye 27(8):972CrossRefPubMedPubMedCentral
2.
go back to reference Ueta T, Yamaguchi Y, Shirakawa Y, Nakano T, Ideta R, Noda Y, Morita A, Mochizuki R, Sugita N, Mitsuishi M, Tamaki Y (2009) Robot-assisted vitreoretinal surgery: development of a prototype and feasibility studies in an animal model. Ophthalmology 116(8):1538–1543CrossRefPubMed Ueta T, Yamaguchi Y, Shirakawa Y, Nakano T, Ideta R, Noda Y, Morita A, Mochizuki R, Sugita N, Mitsuishi M, Tamaki Y (2009) Robot-assisted vitreoretinal surgery: development of a prototype and feasibility studies in an animal model. Ophthalmology 116(8):1538–1543CrossRefPubMed
3.
go back to reference He C, Huang L, Yang Y, Liang Q, Li Y (2018) Research and realization of a master–slave robotic system for retinal vascular bypass surgery. Chin J Mech Eng 31(1):78CrossRef He C, Huang L, Yang Y, Liang Q, Li Y (2018) Research and realization of a master–slave robotic system for retinal vascular bypass surgery. Chin J Mech Eng 31(1):78CrossRef
4.
go back to reference MacLachlan RA, Becker BC, Tabarés JC, Podnar GW, Lobes LA Jr, Riviere CN (2012) Micron: an actively stabilized handheld tool for microsurgery. IEEE Trans Robot 28(1):195–212CrossRefPubMed MacLachlan RA, Becker BC, Tabarés JC, Podnar GW, Lobes LA Jr, Riviere CN (2012) Micron: an actively stabilized handheld tool for microsurgery. IEEE Trans Robot 28(1):195–212CrossRefPubMed
5.
go back to reference Kummer MP, Abbott JJ, Kratochvil BE, Borer R, Sengul A, Nelson BJ (2010) Octomag: an electromagnetic system for 5-dof wireless micromanipulation. IEEE Trans Robot 26(6):1006–1017CrossRef Kummer MP, Abbott JJ, Kratochvil BE, Borer R, Sengul A, Nelson BJ (2010) Octomag: an electromagnetic system for 5-dof wireless micromanipulation. IEEE Trans Robot 26(6):1006–1017CrossRef
6.
go back to reference Hubschman J, Bourges J, Choi W, Mozayan A, Tsirbas A, Kim C, Schwartz S (2010) The microhand: a new concept of micro-forceps for ocular robotic surgery. Eye 24(2):364CrossRefPubMed Hubschman J, Bourges J, Choi W, Mozayan A, Tsirbas A, Kim C, Schwartz S (2010) The microhand: a new concept of micro-forceps for ocular robotic surgery. Eye 24(2):364CrossRefPubMed
7.
go back to reference Edwards T, Xue K, Meenink H, Beelen M, Naus G, Simunovic M, Latasiewicz M, Farmery A, de Smet M, MacLaren R (2018) First-in-human study of the safety and viability of intraocular robotic surgery. Nat Biomed Eng 2(9):249CrossRef Edwards T, Xue K, Meenink H, Beelen M, Naus G, Simunovic M, Latasiewicz M, Farmery A, de Smet M, MacLaren R (2018) First-in-human study of the safety and viability of intraocular robotic surgery. Nat Biomed Eng 2(9):249CrossRef
8.
go back to reference Gijbels A, Smits J, Schoevaerdts L, Willekens K, Vander Poorten EB, Stalmans P, Reynaerts D (2018) In-human robot-assisted retinal vein cannulation, a world first. Ann Biomed Eng 46(10):1676–1685CrossRefPubMed Gijbels A, Smits J, Schoevaerdts L, Willekens K, Vander Poorten EB, Stalmans P, Reynaerts D (2018) In-human robot-assisted retinal vein cannulation, a world first. Ann Biomed Eng 46(10):1676–1685CrossRefPubMed
9.
go back to reference Üneri A, Balicki MA, Handa J, Gehlbach P, Taylor RH, Iordachita I (2010) New steady-hand eye robot with micro-force sensing for vitreoretinal surgery. In: 2010 3rd IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics (BioRob). IEEE, pp 814–819 Üneri A, Balicki MA, Handa J, Gehlbach P, Taylor RH, Iordachita I (2010) New steady-hand eye robot with micro-force sensing for vitreoretinal surgery. In: 2010 3rd IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics (BioRob). IEEE, pp 814–819
10.
go back to reference Iordachita I, Sun Z, Balicki M, Kang JU, Phee SJ, Handa J, Gehlbach P, Taylor R (2009) A sub-millimetric, 0.25 mn resolution fully integrated fiber-optic force-sensing tool for retinal microsurgery. Int J Comput Assist Radiol Surg 4(4):383–390CrossRefPubMedPubMedCentral Iordachita I, Sun Z, Balicki M, Kang JU, Phee SJ, Handa J, Gehlbach P, Taylor R (2009) A sub-millimetric, 0.25 mn resolution fully integrated fiber-optic force-sensing tool for retinal microsurgery. Int J Comput Assist Radiol Surg 4(4):383–390CrossRefPubMedPubMedCentral
11.
go back to reference He X, Handa J, Gehlbach P, Taylor R, Iordachita I (2014) A submillimetric 3-dof force sensing instrument with integrated fiber bragg grating for retinal microsurgery. IEEE Trans Biomed Eng 61(2):522–534CrossRefPubMedPubMedCentral He X, Handa J, Gehlbach P, Taylor R, Iordachita I (2014) A submillimetric 3-dof force sensing instrument with integrated fiber bragg grating for retinal microsurgery. IEEE Trans Biomed Eng 61(2):522–534CrossRefPubMedPubMedCentral
12.
go back to reference He X, Balicki M, Gehlbach P, Handa J, Taylor R, Iordachita I (2014) A multi-function force sensing instrument for variable admittance robot control in retinal microsurgery. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1411–1418 He X, Balicki M, Gehlbach P, Handa J, Taylor R, Iordachita I (2014) A multi-function force sensing instrument for variable admittance robot control in retinal microsurgery. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1411–1418
13.
go back to reference Cutler N, Balicki M, Finkelstein M, Wang J, Gehlbach P, McGready J, Iordachita I, Taylor R, Handa JT (2013) Auditory force feedback substitution improves surgical precision during simulated ophthalmic surgery. Investig Ophthalmol Vis Sci 54(2):1316–1324CrossRef Cutler N, Balicki M, Finkelstein M, Wang J, Gehlbach P, McGready J, Iordachita I, Taylor R, Handa JT (2013) Auditory force feedback substitution improves surgical precision during simulated ophthalmic surgery. Investig Ophthalmol Vis Sci 54(2):1316–1324CrossRef
14.
go back to reference A. Ebrahimi, C. He, M. Roizenblatt, Patel, S. Niravkumar, Sefati, P. Gehlbach, and I. Iordachita (2018) Real-time sclera force feedback for enabling safe robot assisted vitreoretinal surgery. In: 40th Annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 3650–3655 A. Ebrahimi, C. He, M. Roizenblatt, Patel, S. Niravkumar, Sefati, P. Gehlbach, and I. Iordachita (2018) Real-time sclera force feedback for enabling safe robot assisted vitreoretinal surgery. In: 40th Annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 3650–3655
15.
go back to reference Othonos A, Kalli K, Pureur D, Mugnier A (2006) Fibre Bragg gratings. In: Venghaus H (ed) Wavelength filters in fibre optics. Springer, Berlin, pp 189–269CrossRef Othonos A, Kalli K, Pureur D, Mugnier A (2006) Fibre Bragg gratings. In: Venghaus H (ed) Wavelength filters in fibre optics. Springer, Berlin, pp 189–269CrossRef
16.
17.
18.
go back to reference Stollenga MF, Byeon W, Liwicki M, Schmidhuber J (2015) Parallel multi-dimensional lSTM, with application to fast biomedical volumetric image segmentation. In: Advances in neural information processing systems, pp 2998–3006 Stollenga MF, Byeon W, Liwicki M, Schmidhuber J (2015) Parallel multi-dimensional lSTM, with application to fast biomedical volumetric image segmentation. In: Advances in neural information processing systems, pp 2998–3006
19.
go back to reference Sundermeyer M, Schlüter R, Ney H (2012) LSTM neural networks for language modeling. In: Thirteenth annual conference of the international speech communication association Sundermeyer M, Schlüter R, Ney H (2012) LSTM neural networks for language modeling. In: Thirteenth annual conference of the international speech communication association
20.
go back to reference Horise Y, He X, Gehlbach P, Taylor R, Iordachita I (2015) FBG-based sensorized light pipe for robotic intraocular illumination facilitates bimanual retinal microsurgery. In: 37th Annual international conference of the IEEE engineering in medicine and biology society (EMBC) (2015). IEEE, pp 13–16 Horise Y, He X, Gehlbach P, Taylor R, Iordachita I (2015) FBG-based sensorized light pipe for robotic intraocular illumination facilitates bimanual retinal microsurgery. In: 37th Annual international conference of the IEEE engineering in medicine and biology society (EMBC) (2015). IEEE, pp 13–16
21.
go back to reference Kumar R, Berkelman P, Gupta P, Barnes A, Jensen PS, Whitcomb LL, Taylor RH (2000) Preliminary experiments in cooperative human/robot force control for robot assisted microsurgical manipulation. In: IEEE international conference on robotics and automation, 2000. Proceedings. ICRA’00, vol 1. IEEE, pp 610–617 Kumar R, Berkelman P, Gupta P, Barnes A, Jensen PS, Whitcomb LL, Taylor RH (2000) Preliminary experiments in cooperative human/robot force control for robot assisted microsurgical manipulation. In: IEEE international conference on robotics and automation, 2000. Proceedings. ICRA’00, vol 1. IEEE, pp 610–617
22.
go back to reference Greff K, Srivastava RK, Koutník J, Steunebrink BR, Schmidhuber J (2017) LSTM: a search space odyssey. IEEE Trans Neural Netw Learn Syst 28(10):2222–2232CrossRefPubMed Greff K, Srivastava RK, Koutník J, Steunebrink BR, Schmidhuber J (2017) LSTM: a search space odyssey. IEEE Trans Neural Netw Learn Syst 28(10):2222–2232CrossRefPubMed
25.
go back to reference He C, Ebrahimi A, Roizenblatt M, Patel N, Yang Y, Gehlbach PL, Iordachita I (2018) User behavior evaluation in robot-assisted retinal surgery. In: 2018 27th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 174–179 He C, Ebrahimi A, Roizenblatt M, Patel N, Yang Y, Gehlbach PL, Iordachita I (2018) User behavior evaluation in robot-assisted retinal surgery. In: 2018 27th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 174–179
Metadata
Title
Preliminary study of an RNN-based active interventional robotic system (AIRS) in retinal microsurgery
Authors
Changyan He
Niravkumar Patel
Ali Ebrahimi
Marin Kobilarov
Iulian Iordachita
Publication date
01-06-2019
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 6/2019
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-019-01947-9

Other articles of this Issue 6/2019

International Journal of Computer Assisted Radiology and Surgery 6/2019 Go to the issue