Skip to main content
Top
Published in: Surgical Endoscopy 6/2014

01-06-2014 | New Technology

Kinect technology for hand tracking control of surgical robots: technical and surgical skill comparison to current robotic masters

Authors: Yonjae Kim, Simon Leonard, Azad Shademan, Axel Krieger, Peter C. W. Kim

Published in: Surgical Endoscopy | Issue 6/2014

Login to get access

Abstract

Background

Current surgical robots are controlled by a mechanical master located away from the patient, tracking surgeon’s hands by wire and pulleys or mechanical linkage. Contactless hand tracking for surgical robot control is an attractive alternative, because it can be executed with minimal footprint at the patient’s bedside without impairing sterility, while eliminating current disassociation between surgeon and patient. We compared technical and technologic feasibility of contactless hand tracking to the current clinical standard master controllers.

Methods

A hand-tracking system (Kinect™-based 3Gear), a wire-based mechanical master (Mantis Duo), and a clinical mechanical linkage master (da Vinci) were evaluated for technical parameters with strong clinical relevance: system latency, static noise, robot slave tremor, and controller range. Five experienced surgeons performed a skill comparison study, evaluating the three different master controllers for efficiency and accuracy in peg transfer and pointing tasks.

Results

da Vinci had the lowest latency of 89 ms, followed by Mantis with 374 ms and 3Gear with 576 ms. Mantis and da Vinci produced zero static error. 3Gear produced average static error of 0.49 mm. The tremor of the robot used by the 3Gear and Mantis system had a radius of 1.7 mm compared with 0.5 mm for da Vinci. The three master controllers all had similar range. The surgeons took 1.98 times longer to complete the peg transfer task with the 3Gear system compared with Mantis, and 2.72 times longer with Mantis compared with da Vinci (p value 2.1e−9). For the pointer task, surgeons were most accurate with da Vinci with average error of 0.72 mm compared with Mantis’s 1.61 mm and 3Gear’s 2.41 mm (p value 0.00078).

Conclusions

Contactless hand-tracking technology as a surgical master can execute simple surgical tasks. Whereas traditional master controllers outperformed, given that contactless hand-tracking is a first-generation technology, clinical potential is promising and could become a reality with some technical improvements.
Literature
1.
go back to reference Tsui C, Kelin R, Garabrant M (2013) Minimally invasive surgery: national trends in adoption and future directions for hospital strategy. Surg Endosc 27:2253–2257PubMedCrossRef Tsui C, Kelin R, Garabrant M (2013) Minimally invasive surgery: national trends in adoption and future directions for hospital strategy. Surg Endosc 27:2253–2257PubMedCrossRef
2.
go back to reference Wetter PA, Kavic MS, Levinson CJ, Kelley WE, McDougall EM, Nezhat C (2005) Prevention and management of laparoendoscopic surgical complications, 2nd edn. Society of Laparoscopic Surgeons, Miami Wetter PA, Kavic MS, Levinson CJ, Kelley WE, McDougall EM, Nezhat C (2005) Prevention and management of laparoendoscopic surgical complications, 2nd edn. Society of Laparoscopic Surgeons, Miami
4.
6.
go back to reference Jacob MG, Wachs JP, Packer RA (2013) Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images. J Am Med Inform Assoc. doi:10.1136/amiajnl-2012-001212 PubMed Jacob MG, Wachs JP, Packer RA (2013) Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images. J Am Med Inform Assoc. doi:10.​1136/​amiajnl-2012-001212 PubMed
7.
go back to reference Van den Bergh M, Carton D, de Nijs R, Mitsou N, Landsiedel C, Kuehnlenz K, Wollherr D (2011) Real-time 3D hand gesture interaction with a robot for understanding directions from humans. IEEE RO-MAN pp 357–362 doi:10.1109/ROMAN.2011.6005195 July 31, 2011 Van den Bergh M, Carton D, de Nijs R, Mitsou N, Landsiedel C, Kuehnlenz K, Wollherr D (2011) Real-time 3D hand gesture interaction with a robot for understanding directions from humans. IEEE RO-MAN pp 357–362 doi:10.​1109/​ROMAN.​2011.​6005195 July 31, 2011
10.
go back to reference Oikonomidis I, Kyriazis N, Argyros A (2011) Efficient model-based 3D tracking of hand articulations using Kinect Proceedings of the British Machine Vision Conference. doi:10.5244/C.25.101, Sep 2011 Oikonomidis I, Kyriazis N, Argyros A (2011) Efficient model-based 3D tracking of hand articulations using Kinect Proceedings of the British Machine Vision Conference. doi:10.​5244/​C.​25.​101, Sep 2011
11.
go back to reference Jacob MG, Li YT, Wachs JP (2011) A gesture driven robotic scrub nurse. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp 2039–2044. doi:10.1109/ICSMC.2011.6083972, Oct 2011 Jacob MG, Li YT, Wachs JP (2011) A gesture driven robotic scrub nurse. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp 2039–2044. doi:10.​1109/​ICSMC.​2011.​6083972, Oct 2011
12.
go back to reference Monnich H, Nicolai P, Beyl T, Raczkowsky J, Worn H (2011) A supervision system for the intuitive usage of a telemanipulated surgical robotic setup. IEEE Int Conf Robot Biomimetics ROBIO, pp 449–454 doi:10.1109/ROBIO.2011.6181327 Dec 2011 Monnich H, Nicolai P, Beyl T, Raczkowsky J, Worn H (2011) A supervision system for the intuitive usage of a telemanipulated surgical robotic setup. IEEE Int Conf Robot Biomimetics ROBIO, pp 449–454 doi:10.​1109/​ROBIO.​2011.​6181327 Dec 2011
14.
go back to reference Lum MJ, Rosen J, King H, Friedman DC, Lendvay TS, Wright AS, Sinanan MN, Hannaford B (2009) Teleoperation in surgical robotics – network latency effects on surgical performance. Annu Int Conf IEEE Eng Med Biol Soc, Minneapolis, pp 6860–6863 doi:10.1109/IEMBS.2009.5333120, Sept 2009 Lum MJ, Rosen J, King H, Friedman DC, Lendvay TS, Wright AS, Sinanan MN, Hannaford B (2009) Teleoperation in surgical robotics – network latency effects on surgical performance. Annu Int Conf IEEE Eng Med Biol Soc, Minneapolis, pp 6860–6863 doi:10.​1109/​IEMBS.​2009.​5333120, Sept 2009
Metadata
Title
Kinect technology for hand tracking control of surgical robots: technical and surgical skill comparison to current robotic masters
Authors
Yonjae Kim
Simon Leonard
Azad Shademan
Axel Krieger
Peter C. W. Kim
Publication date
01-06-2014
Publisher
Springer US
Published in
Surgical Endoscopy / Issue 6/2014
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-013-3383-8

Other articles of this Issue 6/2014

Surgical Endoscopy 6/2014 Go to the issue