Skip to main content
Top

01-08-2012 | Review

Review of surgical robotics user interface: what is the best way to control robotic surgery?

Authors: Anton Simorov, R. Stephen Otte, Courtni M. Kopietz, Dmitry Oleynikov

Published in: Surgical Endoscopy | Issue 8/2012

Login to get access

Abstract

Background

As surgical robots begin to occupy a larger place in operating rooms around the world, continued innovation is necessary to improve our outcomes.

Methods

A comprehensive review of current surgical robotic user interfaces was performed to describe the modern surgical platforms, identify the benefits, and address the issues of feedback and limitations of visualization.

Results

Most robots currently used in surgery employ a master/slave relationship, with the surgeon seated at a work-console, manipulating the master system and visualizing the operation on a video screen. Although enormous strides have been made to advance current technology to the point of clinical use, limitations still exist. A lack of haptic feedback to the surgeon and the inability of the surgeon to be stationed at the operating table are the most notable examples. The future of robotic surgery sees a marked increase in the visualization technologies used in the operating room, as well as in the robots’ abilities to convey haptic feedback to the surgeon. This will allow unparalleled sensation for the surgeon and almost eliminate inadvertent tissue contact and injury.

Conclusions

A novel design for a user interface will allow the surgeon to have access to the patient bedside, remaining sterile throughout the procedure, employ a head-mounted three-dimensional visualization system, and allow the most intuitive master manipulation of the slave robot to date.
Literature
3.
4.
go back to reference Shah A, Okotie OT, Zhao L, Pins MR, Bhalani V, Dalton DP (2008) Pathologic outcomes during the learning curve for robotic-assisted laparoscopic radical prostatectomy. Int Braz J Urol 34(2):159–163PubMedCrossRef Shah A, Okotie OT, Zhao L, Pins MR, Bhalani V, Dalton DP (2008) Pathologic outcomes during the learning curve for robotic-assisted laparoscopic radical prostatectomy. Int Braz J Urol 34(2):159–163PubMedCrossRef
5.
go back to reference Tan GY, Goel RK, Kaouk JH, Tewari AK (2009) Technological advances in robotic-assisted laparoscopic surgery. Urol Clin North Am 36(2):237–249PubMedCrossRef Tan GY, Goel RK, Kaouk JH, Tewari AK (2009) Technological advances in robotic-assisted laparoscopic surgery. Urol Clin North Am 36(2):237–249PubMedCrossRef
6.
go back to reference Phee SJ, Low SC, Huynh VA, Kencana AP, Sun ZL, Yang K (2009) Master and slave transluminal endoscopic robot (MASTER) for natural orifice transluminal endoscopic surgery (NOTES). Conf Proc IEEE Eng Med Biol Soc 2009:1192–1195PubMed Phee SJ, Low SC, Huynh VA, Kencana AP, Sun ZL, Yang K (2009) Master and slave transluminal endoscopic robot (MASTER) for natural orifice transluminal endoscopic surgery (NOTES). Conf Proc IEEE Eng Med Biol Soc 2009:1192–1195PubMed
7.
go back to reference Sun Z, Ang RY, Lim EW, Wang Z, Ho KY, Phee SJ (2011) Enhancement of a master-slave robotic system for natural orifice transluminal endoscopic surgery. Ann Acad Med Singapore 40(5):223–228PubMed Sun Z, Ang RY, Lim EW, Wang Z, Ho KY, Phee SJ (2011) Enhancement of a master-slave robotic system for natural orifice transluminal endoscopic surgery. Ann Acad Med Singapore 40(5):223–228PubMed
8.
go back to reference Phee SJ, Ho KY, Lomanto D, Low SC, Huynh VA, Kencana AP, Yang K, Sun ZL, Chung SC (2010) Natural orifice transgastric endoscopic wedge hepatic resection in an experimental model using an intuitively controlled master and slave transluminal endoscopic robot (MASTER). Surg Endosc 24(9):2293–2298PubMedCrossRef Phee SJ, Ho KY, Lomanto D, Low SC, Huynh VA, Kencana AP, Yang K, Sun ZL, Chung SC (2010) Natural orifice transgastric endoscopic wedge hepatic resection in an experimental model using an intuitively controlled master and slave transluminal endoscopic robot (MASTER). Surg Endosc 24(9):2293–2298PubMedCrossRef
9.
go back to reference Ho KY, Phee SJ, Shabbir A, Low SC, Huynh VA, Kencana AP, Yang K, Lomanto D, So BY, Wong YY, Chung SC (2010) Endoscopic submucosal dissection of gastric lesions by using a Master and Slave Transluminal Endoscopic Robot (MASTER). Gastrointest Endosc 72(3):593–599PubMedCrossRef Ho KY, Phee SJ, Shabbir A, Low SC, Huynh VA, Kencana AP, Yang K, Lomanto D, So BY, Wong YY, Chung SC (2010) Endoscopic submucosal dissection of gastric lesions by using a Master and Slave Transluminal Endoscopic Robot (MASTER). Gastrointest Endosc 72(3):593–599PubMedCrossRef
10.
go back to reference Hagn U, Konietschke R, Tobergte A, Nickl M, Jörg S, Kübler B, Passig G, Gröger M, Fröhlich F, Seibold U, Le-Tien L, Albu-Schäffer A, Nothhelfer A, Hacker F, Grebenstein M, Hirzinger G (2010) DLR MiroSurge: a versatile system for research in endoscopic telesurgery. Int J Comput Assist Radiol Surg 5(2):183–193PubMedCrossRef Hagn U, Konietschke R, Tobergte A, Nickl M, Jörg S, Kübler B, Passig G, Gröger M, Fröhlich F, Seibold U, Le-Tien L, Albu-Schäffer A, Nothhelfer A, Hacker F, Grebenstein M, Hirzinger G (2010) DLR MiroSurge: a versatile system for research in endoscopic telesurgery. Int J Comput Assist Radiol Surg 5(2):183–193PubMedCrossRef
12.
go back to reference Konietschke R, Hagn U, Nickl M, Jörg S, Tobergte A, Passig G, Seibold U, Le Tien L, Kuebler B, Gröger M, Fröhlich F, Rink C, Albu-Schäffer A, Grebenstein M, Ortmaier T, Hirzinger G (2009) The DLR MiroSurge: a robotic system for surgery. Video contribution presented at ICRA Konietschke R, Hagn U, Nickl M, Jörg S, Tobergte A, Passig G, Seibold U, Le Tien L, Kuebler B, Gröger M, Fröhlich F, Rink C, Albu-Schäffer A, Grebenstein M, Ortmaier T, Hirzinger G (2009) The DLR MiroSurge: a robotic system for surgery. Video contribution presented at ICRA
13.
go back to reference Suppa M, Kielhofer S, Langwald J, Hacker F, Strobl KH, Hirzinger G (2007) The 3D-modeller: a multi-purpose vision platform. IEEE Int Conf Robot Autom 2007:781–787. doi:10.1109/ROBOT.2007.363081 Suppa M, Kielhofer S, Langwald J, Hacker F, Strobl KH, Hirzinger G (2007) The 3D-modeller: a multi-purpose vision platform. IEEE Int Conf Robot Autom 2007:781–787. doi:10.​1109/​ROBOT.​2007.​363081
14.
go back to reference Harnett BM, Doarn CR, Rosen J, Hannaford B, Broderick TJ (2008) Evaluation of unmanned airborne vehicles and mobile robotic telesurgery in an extreme environment. Telemed J E Health 14(6):539–544PubMedCrossRef Harnett BM, Doarn CR, Rosen J, Hannaford B, Broderick TJ (2008) Evaluation of unmanned airborne vehicles and mobile robotic telesurgery in an extreme environment. Telemed J E Health 14(6):539–544PubMedCrossRef
15.
16.
go back to reference Bornhoft JM, Strabala KW, Wortman TD, Lehman AC, Oleynikov D, Farritor SM (2011) Stereoscopic visualization and haptic technology used to create a virtual environment for remote surgery. Biomed Sci Instrum 47:76–81PubMed Bornhoft JM, Strabala KW, Wortman TD, Lehman AC, Oleynikov D, Farritor SM (2011) Stereoscopic visualization and haptic technology used to create a virtual environment for remote surgery. Biomed Sci Instrum 47:76–81PubMed
17.
go back to reference Zhang X, Nelson C, Oleynikov D (2011) Natural haptic interface for single-port surgical robot with gravity compensation. Int J Med Robot 7(S1). Epub 4 Nov 2011 Zhang X, Nelson C, Oleynikov D (2011) Natural haptic interface for single-port surgical robot with gravity compensation. Int J Med Robot 7(S1). Epub 4 Nov 2011
18.
go back to reference Lum M, Friedman D, Rosen J et al (2009) The RAVEN–design and validation of a telesurgery system. Int J Rob Res 28(9):1183–1197CrossRef Lum M, Friedman D, Rosen J et al (2009) The RAVEN–design and validation of a telesurgery system. Int J Rob Res 28(9):1183–1197CrossRef
19.
go back to reference Lang MJ, Greer AD, Sutherland GR (2011) Intra-operative robotics: NeuroArm. Acta Neurochir Suppl 109:231–236PubMedCrossRef Lang MJ, Greer AD, Sutherland GR (2011) Intra-operative robotics: NeuroArm. Acta Neurochir Suppl 109:231–236PubMedCrossRef
20.
go back to reference Samad MD, Hu Y, Sutherland GR (2010) Effect of force feedback from each DOF on the motion accuracy of a surgical tool in performing a robot-assisted tracing task. Conf Proc IEEE Eng Med Biol Soc 2010:2093–2096PubMed Samad MD, Hu Y, Sutherland GR (2010) Effect of force feedback from each DOF on the motion accuracy of a surgical tool in performing a robot-assisted tracing task. Conf Proc IEEE Eng Med Biol Soc 2010:2093–2096PubMed
21.
go back to reference Sutherland GR, Latour I, Greer AD (2008) Integrating an image-guided robot with intraoperative MRI. IEEE Eng Med Biol Mag 27(3):59–65PubMedCrossRef Sutherland GR, Latour I, Greer AD (2008) Integrating an image-guided robot with intraoperative MRI. IEEE Eng Med Biol Mag 27(3):59–65PubMedCrossRef
22.
go back to reference Joel P, Rosen J, Burns S (2007) Upper-limb powered exoskeleton design. IEEE ASME Trans Mechatron 12(4):408–417CrossRef Joel P, Rosen J, Burns S (2007) Upper-limb powered exoskeleton design. IEEE ASME Trans Mechatron 12(4):408–417CrossRef
23.
go back to reference Joel P, Powell J, Rosen J (2009) Isotropy of an upper limb exoskeleton and the kinematics and dynamics of the human arm. Appl Bionics Biomech 6(2):175–191CrossRef Joel P, Powell J, Rosen J (2009) Isotropy of an upper limb exoskeleton and the kinematics and dynamics of the human arm. Appl Bionics Biomech 6(2):175–191CrossRef
24.
go back to reference Ettore C, Rosen J, Joel P, Burns S (2006) Myoprocessor for neural controlled powered exoskeleton arm. IEEE Trans Biomed Eng 53(11):2387–2396CrossRef Ettore C, Rosen J, Joel P, Burns S (2006) Myoprocessor for neural controlled powered exoskeleton arm. IEEE Trans Biomed Eng 53(11):2387–2396CrossRef
25.
go back to reference Wortman TD, Strabala KW, Lehman AC, Farritor SM, Oleynikov D (2011) Laparoendoscopic single-site surgery using a multi-functional miniature in vivo robot. Int J Med Robot 7(1):17–21PubMedCrossRef Wortman TD, Strabala KW, Lehman AC, Farritor SM, Oleynikov D (2011) Laparoendoscopic single-site surgery using a multi-functional miniature in vivo robot. Int J Med Robot 7(1):17–21PubMedCrossRef
26.
go back to reference Lehman A, Wood N, Farritor S, Goede M, Oleynikov D (2011) Dexterous miniature robot for advanced minimally invasive surgery. Surg Endosc 25(1):119–123PubMedCrossRef Lehman A, Wood N, Farritor S, Goede M, Oleynikov D (2011) Dexterous miniature robot for advanced minimally invasive surgery. Surg Endosc 25(1):119–123PubMedCrossRef
27.
go back to reference Teber D, Baumhauer M, Guven EO et al (2009) Robotic and imaging in urological surgery. Curr Opin Urol 19:108–113PubMedCrossRef Teber D, Baumhauer M, Guven EO et al (2009) Robotic and imaging in urological surgery. Curr Opin Urol 19:108–113PubMedCrossRef
28.
go back to reference Pamplona VF, Fernandes LAF, Prauchner J, Nedel LP, Olivier MM (2008) The image-based data glove. Proceedings of X Symposium on Virtual Real (SVR 2008) 204–211 Pamplona VF, Fernandes LAF, Prauchner J, Nedel LP, Olivier MM (2008) The image-based data glove. Proceedings of X Symposium on Virtual Real (SVR 2008) 204–211
29.
go back to reference Sturman DJ, Zeltzer D (1994) A survey of glove based-input. IEEE Comput Graph Appl 14(1):30–39CrossRef Sturman DJ, Zeltzer D (1994) A survey of glove based-input. IEEE Comput Graph Appl 14(1):30–39CrossRef
30.
go back to reference Faraz A, Payandeh S (2000) Engineering approaches to mechanical and robotic design for minimally invasive surgery (MIS). Kluwer Academic Publishers, Boston, pp 1–11 Faraz A, Payandeh S (2000) Engineering approaches to mechanical and robotic design for minimally invasive surgery (MIS). Kluwer Academic Publishers, Boston, pp 1–11
31.
go back to reference Okamura AM (2009) Haptic feedback in robot-assisted minimally invasive surgery. Curr Opin Urol 19(1):102–107PubMedCrossRef Okamura AM (2009) Haptic feedback in robot-assisted minimally invasive surgery. Curr Opin Urol 19(1):102–107PubMedCrossRef
32.
go back to reference Tholey G, Desai JP (2007) A general purpose 7 DOF haptic device: applications towards robot-assisted surgery. IEEE/ASME Trans Mech 12(6):662–669CrossRef Tholey G, Desai JP (2007) A general purpose 7 DOF haptic device: applications towards robot-assisted surgery. IEEE/ASME Trans Mech 12(6):662–669CrossRef
33.
go back to reference Mavash M (2006) Novel approach for modeling separating forces between deformable bodies. IEEE Trans Inf Technol Biomed 10(3):618–626CrossRef Mavash M (2006) Novel approach for modeling separating forces between deformable bodies. IEEE Trans Inf Technol Biomed 10(3):618–626CrossRef
34.
go back to reference Mavash M, Hayward V (2004) High fidelity haptic synthesis of contact with deformable bodies. IEEE Comput Graph Appl 24(2):28–55 Mavash M, Hayward V (2004) High fidelity haptic synthesis of contact with deformable bodies. IEEE Comput Graph Appl 24(2):28–55
35.
go back to reference Mavash M, Voo LM, Kim D et al (2008) Modeling the forces of cutting with scissors. IEEE Trans Biomed Eng 55(3):848–856CrossRef Mavash M, Voo LM, Kim D et al (2008) Modeling the forces of cutting with scissors. IEEE Trans Biomed Eng 55(3):848–856CrossRef
36.
go back to reference Weiss H, Ortmaier T, Maass H et al (2003) A virtual reality based haptic surgical training system. Comput Aided Surg 8(5):269–272PubMedCrossRef Weiss H, Ortmaier T, Maass H et al (2003) A virtual reality based haptic surgical training system. Comput Aided Surg 8(5):269–272PubMedCrossRef
37.
go back to reference Wagner CR, Howe RD (2007) Force feedback benefit depends on experience in multiple degree of freedom robotic surgery. IEEE Trans Rob 23(6):1235–1240CrossRef Wagner CR, Howe RD (2007) Force feedback benefit depends on experience in multiple degree of freedom robotic surgery. IEEE Trans Rob 23(6):1235–1240CrossRef
Metadata
Title
Review of surgical robotics user interface: what is the best way to control robotic surgery?
Authors
Anton Simorov
R. Stephen Otte
Courtni M. Kopietz
Dmitry Oleynikov
Publication date
01-08-2012
Publisher
Springer-Verlag
Published in
Surgical Endoscopy / Issue 8/2012
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-012-2182-y