Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
International Journal of Computer Assisted Radiology and Surgery
Published in: International Journal of Computer Assisted Radiology and Surgery 12/2020

01-12-2020 | Minimally Invasive Surgery | Original Article

Vision-based hand–eye calibration for robot-assisted minimally invasive surgery

Authors: Yanwen Sun, Bo Pan, Yongchen Guo, Yili Fu, Guojun Niu

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 12/2020

Login to get access

Abstract

Purpose

The knowledge of laparoscope vision can greatly improve the surgical operation room (OR) efficiency. For the vision-based computer-assisted surgery, the hand–eye calibration establishes the coordinate relationship between laparoscope and robot slave arm. While significant advances have been made for hand–eye calibration in recent years, efficient algorithm for minimally invasive surgical robot is still a major challenge. Removing the external calibration object in abdominal environment to estimate the hand–eye transformation is still a critical problem.

Methods

We propose a novel hand–eye calibration algorithm to tackle the problem which relies purely on surgical instrument already in the operating scenario for robot-assisted minimally invasive surgery (RMIS). Our model is formed by the geometry information of the surgical instrument and the remote center-of-motion (RCM) constraint. We also enhance the algorithm with stereo laparoscope model.

Results

Promising validation of synthetic simulation and experimental surgical robot system have been conducted to evaluate the proposed method. We report results that the proposed method can exhibit the hand–eye calibration without calibration object.

Conclusion

Vision-based hand–eye calibration is developed. We demonstrate the feasibility to perform hand–eye calibration by taking advantage of the components of surgical robot system, leading to the efficiency of surgical OR.
Literature
1.
Freschi C, Ferrari V, Melfi F, Ferrari M, Mosca F, Cuschieri A (2013) Technical review of the da Vinci surgical telemanipulator. Int J Med Robot Comput Assist Surg 9(4):396–406CrossRef
2.
Su H, Shuai Li, Jagadesh M, Bascetta L, Ferrigno G, De ME (2019) Manipulability optimization control of a serial redundant robot for robot-assisted minimally invasive surgery. In: 2019 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1323–1328
3.
Kassahun Y, Yu B, Tibebu AT, Stoyanov D, Giannarou S, Metzen JH, Poorten EV (2016) Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int J Comput Assist Radiol Surg 11(4):553–568CrossRef
4.
Kehoe B, Kahn G, Mahler J, Kim J, Lee A, Lee A, Nakagawa K, Patil S, Boyd WD, Abbeel P, Goldberg K (2013) Autonomous multilateral debridement with the Raven surgical robot. In: 2013 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1432–1439
5.
Murali A, Sen S, Kehoe B, Garg A, Goldberg K (2015) Learning by observation for surgical subtasks: multilateral cutting of 3D viscoelastic and 2D orthotropic tissue phantoms. In: 2015 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1202–1209
6.
Thananjeyan B, Garg A, Krishnan S, Chen C, Goldberg K (2017) Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioning. In: 2017 IEEE international conference on robotics and automation (ICRA). IEEE, pp 2371–2378
7.
Allan M, Ourselin S, Hawkes DJ, Kelly JD, Stoyanov D (2018) 3-D pose estimation of articulated instruments in robotic minimally invasive surgery. IEEE Trans Med Imaging 37(5):1204–1213CrossRef
8.
Allan M, Chang PL, Ourselin S, Hawkes DJ, Sridhar A, Kelly J, Stoyanov D (2015) Image based surgical instrument pose estimation with multi-class labelling and optical flow. In: 2015 international conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 331-338
9.
Du X, Kurmann T, Chang PL, Allan M, Ourselin S, Sznitman R, Stoyanov D (2018) Articulated multi-instrument 2-D pose estimation using fully convolutional networks. IEEE Trans Med Imaging 37(5):1276–1287CrossRef
10.
Li W, Dong M, Lu N, Lou X, Sun P (2018) Simultaneous robot–world and hand–eye calibration without a calibration object. Sensors 18(11):3949CrossRef
11.
Park FC, Martin BJ (1994) Robot sensor calibration: solving AX = XB on the Euclidean group. IEEE Trans Robot Autom 10(5):717–721CrossRef
12.
Andreff N, Horaud R, Espiau B (1999) On-line hand–eye calibration. In: Second international conference on 3-D digital imaging and modeling (Cat. No. PR00062). IEEE, pp 430–436
13.
Horaud R, Dornaika F (1995) Hand–eye calibration. Int J Robot Res 14(3):195–210CrossRef
14.
Daniilidis K (1999) Hand–eye calibration using dual quaternions. Int J Robot Res 18(3):286–298CrossRef
15.
Zhang Z, Zhang L, Yang GZ (2017) A computationally efficient method for hand–eye calibration. Int J Comput Assist Radiol Surg 12(10):1775–1787CrossRef
16.
Thompson S, Stoyanov D, Schneider C, Gurusamy K, Ourselin S, Davidson B, Clarkson MJ (2016) Hand–eye calibration for rigid laparoscopes using an invariant point. Int J Comput Assist Radiol Surg 11(6):1071–1080CrossRef
17.
Morgan I, Jayarathne U, Rankin A, Peters TM, Chen EC (2017) Hand–eye calibration for surgical cameras: a procrustean perspective-n-point solution. Int J Comput Assist Radiol Surg 12(7):1141–1149CrossRef
18.
Malti A, Barreto J P (2010) Robust hand–eye calibration for computer aided medical endoscopy. In: 2010 IEEE international conference on robotics and automation. IEEE, pp 5543–5549
19.
Pachtrachai K, Vasconcelos F, Dwyer G, Hailes S, Stoyanov D (2019) Hand–eye calibration with a remote centre of motion. IEEE Robot Autom Lett 4(4):3121–3128CrossRef
20.
Andreff N, Horaud R, Espiau B (2001) Robot hand–eye calibration using structure-from-motion. Int J Robot Res 20(3):228–248CrossRef
21.
Pachtrachai K, Allan M, Pawar V, Hailes S, Stoyanov D (2016) Hand–eye calibration for robotic assisted minimally invasive surgery without a calibration object. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 2485–2491
22.
Strobl KH, Hirzinger G (2006) Optimal hand–eye calibration. In: 2006 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 4647–4653
23.
Malti A (2013) Hand–eye calibration with epipolar constraints: application to endoscopy. Robot Autonom Syst 61(2):161–169CrossRef
24.
Malti A, Barreto JP (2013) Hand–eye and radial distortion calibration for rigid endoscopes. Int J Med Robot Comput Assist Surg 9(4):441–454CrossRef
25.
Wang Z, Liu Z, Ma Q, Cheng A, Liu YH, Kim S, Taylor RH (2017) Vision-based calibration of dual RCM-based robot arms in human-robot collaborative minimally invasive surgery. IEEE Robot Autom Lett 3(2):672–679CrossRef
26.
Pachtrachai K, Vasconcelos F, Chadebecq F, Chadebecq F, Allan M, Hailes S, Pawar V, Stoyanov D (2018) Adjoint transformation algorithm for hand–eye calibration with applications in robotic assisted surgery. Ann Biomed Eng 46(10):1606–1620CrossRef
27.
Corke P (2017) Robotics vision and control: fundamental algorithms In MATLAB® second. Springer, BerlinCrossRef
28.
Bouget D, Allan M, Stoyanov D, Jannin P (2017) Vision-based and marker-less surgical tool detection and tracking: a review of the literature. Med Image Anal 35:633–654CrossRef
29.
Kaehler A, Bradski GR (2016) Learning OpenCV 3. O’Reilly Media, Sebastopol
30.
Stockman George C (2001) Computer vision. Prentice Hall, Upper Saddle River
Metadata
Title
Vision-based hand–eye calibration for robot-assisted minimally invasive surgery
Authors
Yanwen Sun
Bo Pan
Yongchen Guo
Yili Fu
Guojun Niu
Publication date
01-12-2020
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 12/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-020-02245-5

Other articles of this Issue 12/2020

International Journal of Computer Assisted Radiology and Surgery 12/2020 Go to the issue

Original Article

Assessing and mitigating the effects of class imbalance in machine learning with application to X-ray imaging

Original Article

Bone age assessment based on deep convolution neural network incorporated with segmentation

Original Article

Unravelling the effect of data augmentation transformations in polyp segmentation

Original Article

Robust endocytoscopic image classification based on higher-order symmetric tensor analysis and multi-scale topological statistics

Original Article

Enhanced registration of ultrasound volumes by segmentation of resection cavity in neurosurgical procedures

Original Article

Motion analysis of the JHU-ISI Gesture and Skill Assessment Working Set using Robotics Video and Motion Assessment Software