Skip to main content
Top
Published in: Journal of Medical Systems 1/2020

01-01-2020 | Systems-Level Quality Improvement

Adaptive Fusion-Based Autonomous Laparoscope Control for Semi-Autonomous Surgery

Authors: Yanwen Sun, Bo Pan, Shuizhong Zou, Yili Fu

Published in: Journal of Medical Systems | Issue 1/2020

Login to get access

Abstract

The purpose of this paper is to develop an autonomous tracking algorithm based on adaptive fusion kinematics method, the autonomous laparoscope control algorithm and adaptive fusion kinematics method are proposed for semi-autonomous surgery, focus on solving the problems of autonomous laparoscope field of view control for surgical robot system. A novel autonomous tracking algorithm is proposed. To realize more robust tracking, an adaptive fusion kinematics method based on fuzzy logic is proposed, the method adaptive associates the kinematics information of surgical robot system and the laparoscope information. The proposed methods are implemented on the laparoscopic minimally invasive surgical robot system which is developed by our laboratory. Two experiments are carried out, the results indicate that the accurate autonomous field of view control is achieved with the addition of laparoscope information, laparoscopic motion frequency is reduced, the methods can avoid the laparoscope continuous motion and ensure the stability of field of view. The proposed methods improve the intelligence level of surgical robot system.
Literature
1.
go back to reference Yang, G. Z., Cambias, J., Cleary, K., Daimler, E., Drake, J., Drake, P. E., Hata, N., Kazanzides, P., Martel, S., Patel, R. V., Santos, V. J., and Taylor, R. H., Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy. Sci. Robot. 2(4):eaam8638, 2017.CrossRef Yang, G. Z., Cambias, J., Cleary, K., Daimler, E., Drake, J., Drake, P. E., Hata, N., Kazanzides, P., Martel, S., Patel, R. V., Santos, V. J., and Taylor, R. H., Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy. Sci. Robot. 2(4):eaam8638, 2017.CrossRef
2.
go back to reference Moustris, G. P., Hiridis, S. C., Deliparaschos, K. M., and Konstantinidis, K. M., Evolution of autonomous and semi-autonomous robotic surgical systems: A review of the literature. Int. J. Med. Robot. Comput. Assist. Surg. 7(4):375–392, 2011.CrossRef Moustris, G. P., Hiridis, S. C., Deliparaschos, K. M., and Konstantinidis, K. M., Evolution of autonomous and semi-autonomous robotic surgical systems: A review of the literature. Int. J. Med. Robot. Comput. Assist. Surg. 7(4):375–392, 2011.CrossRef
3.
go back to reference Kassahun, Y., Yu, B., Tibebu, A. T., Stoyanov, D., Giannarou, S., Metzen, J. H., and Poorten, E. V., Surgical robotics beyond enhanced dexterity instrumentation: A survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int. J. Comput. Assist. Radiol. Surg. 11(4):553–568, 2016.CrossRef Kassahun, Y., Yu, B., Tibebu, A. T., Stoyanov, D., Giannarou, S., Metzen, J. H., and Poorten, E. V., Surgical robotics beyond enhanced dexterity instrumentation: A survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int. J. Comput. Assist. Radiol. Surg. 11(4):553–568, 2016.CrossRef
4.
go back to reference Wijsman, P. J. M., Broeders, I. A. M. J., Brenkman, H. J., Szold, A., and Kaufman, Y., First experience with the autolap™ system: An image-based robotic camera steering device. Surg. Endosc. 32(5):1–7, 2017. Wijsman, P. J. M., Broeders, I. A. M. J., Brenkman, H. J., Szold, A., and Kaufman, Y., First experience with the autolap™ system: An image-based robotic camera steering device. Surg. Endosc. 32(5):1–7, 2017.
5.
go back to reference Voros, S., Haber, G. P., Menudet, J. F., Long, J. A., and Cinquin, P., ViKY robotic scope holder: Initial clinical experience and preliminary results using instrument tracking. IEEE-ASME Trans. Mechatron. 15(6):879–886, 2010. Voros, S., Haber, G. P., Menudet, J. F., Long, J. A., and Cinquin, P., ViKY robotic scope holder: Initial clinical experience and preliminary results using instrument tracking. IEEE-ASME Trans. Mechatron. 15(6):879–886, 2010.
6.
go back to reference Azizian, M., Khoshnam, M., Najmaei, N., and Patel, R. V., Visual servoing in medical robotics: A survey. Part I: Endoscopic and direct vision imaging - techniques and applications. Int. J. Med. Robot. Comput. Assist. Surg. 10(3):263–274, 2014.CrossRef Azizian, M., Khoshnam, M., Najmaei, N., and Patel, R. V., Visual servoing in medical robotics: A survey. Part I: Endoscopic and direct vision imaging - techniques and applications. Int. J. Med. Robot. Comput. Assist. Surg. 10(3):263–274, 2014.CrossRef
7.
go back to reference Rivas-Blanco, I., Lopez-Casado, C., Perez-Del-Pulgar, C. J., Garcia-Vacas, F., Fraile, J. C., and Munoz, V. F., Smart cable-driven camera robotic assistant. IEEE T. Hum.-Mach. Syst. 48(2): 183-196, 2018.CrossRef Rivas-Blanco, I., Lopez-Casado, C., Perez-Del-Pulgar, C. J., Garcia-Vacas, F., Fraile, J. C., and Munoz, V. F., Smart cable-driven camera robotic assistant. IEEE T. Hum.-Mach. Syst. 48(2): 183-196, 2018.CrossRef
8.
go back to reference Weede, O., Monnich, H., Muller, B., and Worn, H. (2011) An intelligent and autonomous endoscopic guidance system for minimally invasive surgery. In: 2011 IEEE international conference on robotics and automation (ICRA). IEEE, pp 5762-5768 Weede, O., Monnich, H., Muller, B., and Worn, H. (2011) An intelligent and autonomous endoscopic guidance system for minimally invasive surgery. In: 2011 IEEE international conference on robotics and automation (ICRA). IEEE, pp 5762-5768
9.
go back to reference Allan, M., Ourselin, S., Thompson, S., Hawkes, D. J., Kelly, J., and Stoyanov, D., Toward detection and localization of instruments in minimally invasive surgery. IEEE Trans. Biomed. Eng. 60(4):1050–1058, 2013.CrossRef Allan, M., Ourselin, S., Thompson, S., Hawkes, D. J., Kelly, J., and Stoyanov, D., Toward detection and localization of instruments in minimally invasive surgery. IEEE Trans. Biomed. Eng. 60(4):1050–1058, 2013.CrossRef
10.
go back to reference Wang, Z., Zi, B., Ding, H., You, W., and Yu, L., Hybrid grey prediction model-based autotracking algorithm for the laparoscopic visual window of surgical robot. Mech. Mach. Theory 123:107–123, 2018.CrossRef Wang, Z., Zi, B., Ding, H., You, W., and Yu, L., Hybrid grey prediction model-based autotracking algorithm for the laparoscopic visual window of surgical robot. Mech. Mach. Theory 123:107–123, 2018.CrossRef
11.
go back to reference Yu, L., Wang, Z., Sun, L., Wang, W., Wang, L., and Du, Z., A new forecasting kinematic algorithm of automatic navigation for a laparoscopic minimally invasive surgical robotic system. Robotica 35(05):1192–1222, 2017.CrossRef Yu, L., Wang, Z., Sun, L., Wang, W., Wang, L., and Du, Z., A new forecasting kinematic algorithm of automatic navigation for a laparoscopic minimally invasive surgical robotic system. Robotica 35(05):1192–1222, 2017.CrossRef
12.
go back to reference Kashyap, S. K., and Raol, J. R., Fuzzy logic applications in filtering and fusion for target tracking. Def. Sci. J. 58(1):120–135, 2008.CrossRef Kashyap, S. K., and Raol, J. R., Fuzzy logic applications in filtering and fusion for target tracking. Def. Sci. J. 58(1):120–135, 2008.CrossRef
13.
go back to reference Sun, S. L., and Deng, Z. L., Multi-sensor optimal information fusion Kalman filter. Automatica 40(6):1017–1023, 2004.CrossRef Sun, S. L., and Deng, Z. L., Multi-sensor optimal information fusion Kalman filter. Automatica 40(6):1017–1023, 2004.CrossRef
14.
go back to reference Raol, J. R., Multi sensor data fusion with MATLAB. Boca Raton: CRC Press, Inc, 2009.CrossRef Raol, J. R., Multi sensor data fusion with MATLAB. Boca Raton: CRC Press, Inc, 2009.CrossRef
15.
go back to reference Ficocelli M, Janabisharifi F (2001) Adaptive filtering for pose estimation in visual servoing. In: 2001 IEEE/RSJ international conference on Intelligent Robots & Systems (IROS). IEEE, pp 19-24 Ficocelli M, Janabisharifi F (2001) Adaptive filtering for pose estimation in visual servoing. In: 2001 IEEE/RSJ international conference on Intelligent Robots & Systems (IROS). IEEE, pp 19-24
16.
go back to reference Lippiello, V., Siciliano, B., and Villani, L., Adaptive extended Kalman filtering for visual motion estimation of 3D objects. Control. Eng. Pract. 15(1):123–134, 2007.CrossRef Lippiello, V., Siciliano, B., and Villani, L., Adaptive extended Kalman filtering for visual motion estimation of 3D objects. Control. Eng. Pract. 15(1):123–134, 2007.CrossRef
17.
go back to reference Baek, Y. M., Tanaka, S., Harada, K., Sugita, N., Morita, A., Sora, S., and Mitsuishi, M., Robust visual tracking of robotic forceps under a microscope using kinematic data fusion. IEEE-ASME Trans. Mechatron. 19(1):278–288, 2014.CrossRef Baek, Y. M., Tanaka, S., Harada, K., Sugita, N., Morita, A., Sora, S., and Mitsuishi, M., Robust visual tracking of robotic forceps under a microscope using kinematic data fusion. IEEE-ASME Trans. Mechatron. 19(1):278–288, 2014.CrossRef
18.
go back to reference Richard, S., Computer vision - algorithms and applications. London: Springer-Verlag, 2011. Richard, S., Computer vision - algorithms and applications. London: Springer-Verlag, 2011.
19.
go back to reference Liu, H., Lai, X., and Wu, W., Time-optimal and jerk-continuous trajectory planning for robot manipulators with kinematic constraints. Robot. Comput. Integr. Manuf. 29(2):309–317, 2013.CrossRef Liu, H., Lai, X., and Wu, W., Time-optimal and jerk-continuous trajectory planning for robot manipulators with kinematic constraints. Robot. Comput. Integr. Manuf. 29(2):309–317, 2013.CrossRef
20.
go back to reference Xiao, Y., Du, Z., and Dong, W., Smooth and near time-optimal trajectory planning of industrial robots for online applications. Ind. Robot 39(2):169–177, 2012.CrossRef Xiao, Y., Du, Z., and Dong, W., Smooth and near time-optimal trajectory planning of industrial robots for online applications. Ind. Robot 39(2):169–177, 2012.CrossRef
21.
go back to reference Ai, Y., Pan, B., Fu, Y., and Wang, S., Design of a novel robotic system for minimally invasive surgery. Ind. Robot 44(3):288–298, 2017.CrossRef Ai, Y., Pan, B., Fu, Y., and Wang, S., Design of a novel robotic system for minimally invasive surgery. Ind. Robot 44(3):288–298, 2017.CrossRef
22.
go back to reference Kaehler, A., and Bradski, G. R., Learning OpenCV 3. Sebastopol: O’Reilly Media, 2016. Kaehler, A., and Bradski, G. R., Learning OpenCV 3. Sebastopol: O’Reilly Media, 2016.
23.
go back to reference Stockman, G. C., Computer vision. Upper Saddle River: Prentice Hall, 2001. Stockman, G. C., Computer vision. Upper Saddle River: Prentice Hall, 2001.
Metadata
Title
Adaptive Fusion-Based Autonomous Laparoscope Control for Semi-Autonomous Surgery
Authors
Yanwen Sun
Bo Pan
Shuizhong Zou
Yili Fu
Publication date
01-01-2020
Publisher
Springer US
Published in
Journal of Medical Systems / Issue 1/2020
Print ISSN: 0148-5598
Electronic ISSN: 1573-689X
DOI
https://doi.org/10.1007/s10916-019-1460-9

Other articles of this Issue 1/2020

Journal of Medical Systems 1/2020 Go to the issue