Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
International Journal of Computer Assisted Radiology and Surgery
Published in: International Journal of Computer Assisted Radiology and Surgery 7/2017

Open Access 01-07-2017 | Original Article

Gaze-contingent perceptually enabled interactions in the operating theatre

Authors: Alexandros A. Kogkas, Ara Darzi, George P. Mylonas

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 7/2017

Login to get access

Abstract

Purpose

Improved surgical outcome and patient safety in the operating theatre are constant challenges. We hypothesise that a framework that collects and utilises information —especially perceptually enabled ones—from multiple sources, could help to meet the above goals. This paper presents some core functionalities of a wider low-cost framework under development that allows perceptually enabled interaction within the surgical environment.

Methods

The synergy of wearable eye-tracking and advanced computer vision methodologies, such as SLAM, is exploited. As a demonstration of one of the framework’s possible functionalities, an articulated collaborative robotic arm and laser pointer is integrated and the set-up is used to project the surgeon’s fixation point in 3D space.

Results

The implementation is evaluated over 60 fixations on predefined targets, with distances between the subject and the targets of 92–212 cm and between the robot and the targets of 42–193 cm. The median overall system error is currently 3.98 cm. Its real-time potential is also highlighted.

Conclusions

The work presented here represents an introduction and preliminary experimental validation of core functionalities of a larger framework under development. The proposed framework is geared towards a safer and more efficient surgical theatre.
Appendix
Available only for authorised users
Literature
1.
2.
Christian CK, Gustafson ML, Roth EM, Sheridan TB, Gandhi TK, Dwyer K, Zinner MJ, Dierks MM (2006) A prospective study of patient safety in the operating room. Surgery 139(2):159–173CrossRefPubMed
3.
Seagull FJ, Moses GR, Park AE (2008) Pillars of a smart, safe operating room. In: Henriksen K, Battles JB, Keyes MA, Grady ML (eds) Performance and tools. Advances in patient safety: new directions and alternative approaches, vol 3. Agency for Healthcare Research and Quality, Rockville, MD
4.
Allan M, Ourselin S, Thompson S, Hawkes DJ, Kelly J, Stoyanov D (2013) Toward detection and localization of instruments in minimally invasive surgery. IEEE Trans Biomed Eng 60(4):1050–1058CrossRefPubMed
5.
Ballard MD (2004) System and method of tracking surgical sponges US Patent 6,777,623
6.
7.
Mewes A, Hensen B, Wacker F, Hansen C (2017) Touchless interaction with software in interventional radiology and surgery: a systematic literature review. Int J Comput Assist Radiol Surg 12(2):291–305CrossRefPubMed
8.
Wachs JP, Jacob M, Li Y-T, Akingba G (2012) Does a robotic scrub nurse improve economy of movements? Proc SPIE Int Soc Opt Eng v8316:83160E
9.
Noonan DP, Mylonas GP, Darzi A, Yang, G-Z (2008) Gaze contingent articulated robot control for robot assisted minimally invasive surgery. In: 2008 IEEE/RSJ international conference on intelligent robots and systems, p 1186–1191
10.
Kwok KW, Sun LW, Mylonas GP, James DRC, Orihuela-Espina F, Yang GZ (2012) Collaborative gaze channelling for improved cooperation during robotic assisted surgery. Ann Biomed Eng 40(10):2156–2167CrossRefPubMedPubMedCentral
11.
Chetwood ASA, Kwok K-W, Sun L-W, Mylonas GP, Clark J, Darzi A, Yang G-Z (2012) Collaborative eye tracking—a potential training tool in laparoscopic surgery. Surg Endosc 26(7):2003–2009CrossRefPubMed
12.
Kogkas AA, Sodergren MH, Darzi A, Mylonas GP (2016) Macro- and micro-scale 3D gaze tracking in the operating theatre. In: The Hamlyn symposium on medical robotics, p 100
13.
Kogkas AA, Darzi A, Mylonas GP (2016) Gaze-driven human–robot interaction in the operating theatre. In: 6th Joint workshop on new technologies for computer/robot assisted surgery (CRAS), p 103
14.
Se S, Lowe D, Little J (2002) Mobile robot localization and mapping with uncertainty using scale-invariant visual landmarks. Robot Res Int J 21(8):735–758CrossRef
15.
16.
Yang L, Zhang L, Dong H, Alelaiwi A, Saddik A (2015) Evaluating and improving the depth accuracy of Kinect for Windows v2. IEEE Sens J 15(8):4275–4285CrossRef
17.
Hansard M, Lee S, Choi O, Horaud R (2013) Time-of-flight cameras: principles, methods and applications. Springer
18.
Zhang C, Zhang Z (2014) Calibration between depth and color sensors for commodity depth cameras. In: Computer Vision and Machine Learning with RGB-D Sensors. Springer, pp 47–64
19.
Lepetit V, Moreno-Noguer F, Fua P (2009) EPnP: an accurate O(n) solution to the PnP problem. Int J Comput Vis 81(2):155–166CrossRef
20.
Miseikis J, Glette K, Elle OJ, Torresen J (2016) Automatic calibration of a robot manipulator and multi 3D camera system. arXiv:​1601.​01566
21.
Rusu RB, Cousins S (2011) 3D is here: point cloud library (PCL). In: Proceedings of IEEE international conference on robotics and automation (ICRA), IEEE, pp 1–4
22.
Marton ZC, Rusu RB, Beetz M (2009) On fast surface reconstruction methods for large and noisy datasets. In: The IEEE international conference on robotics and automation (ICRA’09), IEEE, pp 3218–3223
23.
Mur-Artal R, Montiel JMM, Tardos JD (2015) ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans Robot 31(5):1147–1163CrossRef
24.
Moller T, Trumbore B (1998) Fast, minimum storage ray-triangle intersection. In: ACM SIGGRAPH 2005 Courses, ACM, p 7
25.
Keller M, Lefloch D, Lambers M, Izadi S, Weyrich T, Kolb A (2013) Real-time 3d reconstruction in dynamic scenes using point-based fusion. In: 2013 International conference on 3DTV-conference, pp 1–8
26.
Mardanbegi D, Hansen DW (2012) Parallax error in the monocular head-mounted eye trackers. In: Proceedings of the 2012 ACM conference on ubiquitous computing—UbiComp ’12, p 689
Metadata
Title
Gaze-contingent perceptually enabled interactions in the operating theatre
Authors
Alexandros A. Kogkas
Ara Darzi
George P. Mylonas
Publication date
01-07-2017
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 7/2017
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-017-1580-y

Other articles of this Issue 7/2017

International Journal of Computer Assisted Radiology and Surgery 7/2017 Go to the issue

Original Article

Intelligent viewpoint selection for efficient CT to video registration in laparoscopic liver surgery

Original Article

Deep monocular 3D reconstruction for assisted navigation in bronchoscopy

Original Article

Hand-eye calibration for surgical cameras: a Procrustean Perspective-n-Point solution

Original Article

Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery

Original Article

Can real-time RGBD enhance intraoperative Cone-Beam CT?

Original Article

Evaluation of a novel multi-articulated endoscope: proof of concept through a virtual simulation