Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
Surgical Endoscopy
Published in: Surgical Endoscopy 10/2020

Open Access 01-10-2020 | Liver Surgery | New Technology

Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study

Authors: C. Schneider, S. Thompson, J. Totz, Y. Song, M. Allam, M. H. Sodergren, A. E. Desjardins, D. Barratt, S. Ourselin, K. Gurusamy, D. Stoyanov, M. J. Clarkson, D. J. Hawkes, B. R. Davidson

Published in: Surgical Endoscopy | Issue 10/2020

Login to get access

Abstract

Background

The laparoscopic approach to liver resection may reduce morbidity and hospital stay. However, uptake has been slow due to concerns about patient safety and oncological radicality. Image guidance systems may improve patient safety by enabling 3D visualisation of critical intra- and extrahepatic structures. Current systems suffer from non-intuitive visualisation and a complicated setup process. A novel image guidance system (SmartLiver), offering augmented reality visualisation and semi-automatic registration has been developed to address these issues. A clinical feasibility study evaluated the performance and usability of SmartLiver with either manual or semi-automatic registration.

Methods

Intraoperative image guidance data were recorded and analysed in patients undergoing laparoscopic liver resection or cancer staging. Stereoscopic surface reconstruction and iterative closest point matching facilitated semi-automatic registration. The primary endpoint was defined as successful registration as determined by the operating surgeon. Secondary endpoints were system usability as assessed by a surgeon questionnaire and comparison of manual vs. semi-automatic registration accuracy. Since SmartLiver is still in development no attempt was made to evaluate its impact on perioperative outcomes.

Results

The primary endpoint was achieved in 16 out of 18 patients. Initially semi-automatic registration failed because the IGS could not distinguish the liver surface from surrounding structures. Implementation of a deep learning algorithm enabled the IGS to overcome this issue and facilitate semi-automatic registration. Mean registration accuracy was 10.9 ± 4.2 mm (manual) vs. 13.9 ± 4.4 mm (semi-automatic) (Mean difference − 3 mm; p = 0.158). Surgeon feedback was positive about IGS handling and improved intraoperative orientation but also highlighted the need for a simpler setup process and better integration with laparoscopic ultrasound.

Conclusion

The technical feasibility of using SmartLiver intraoperatively has been demonstrated. With further improvements semi-automatic registration may enhance user friendliness and workflow of SmartLiver. Manual and semi-automatic registration accuracy were comparable but evaluation on a larger patient cohort is required to confirm these findings.
Literature
1.
Wakabayashi G, Cherqui D, Geller DA, Buell JF, Kaneko H, Han HS et al (2015) Recommendations for laparoscopic liver resection. Ann Surg. 261(4):619–629PubMed
2.
Fuks D, Cauchy F, Ftériche S, Nomi T, Schwarz L, Dokmak S et al (2015) Laparoscopy decreases pulmonary complications in patients undergoing major liver resection. Ann Surg 00(00):1
3.
Kirchberg J, Reißfelder C, Weitz J, Koch M (2013) Laparoscopic surgery of liver tumors. Langenbecks Arch Surg 398:931–938PubMed
4.
Ciria R, Cherqui D, Geller DA, Briceno J, Wakabayashi G (2015) Comparative short-term benefits of laparoscopic liver resection. Ann Surg 4:761–777
5.
Nomi T, Fuks D, Kawaguchi Y, Mal F, Nakajima Y, Gayet B (2015) Learning curve for laparoscopic major hepatectomy. Br J Surg 102(7):796–804PubMed
6.
Fusaglia M, Hess H, Schwalbe M, Peterhans M, Tinguely P, Weber S et al (2015) A clinically applicable laser-based image-guided system for laparoscopic liver procedures. Int J Comput Assist Radiol Surg 11:1499–1513PubMed
7.
Kingham TP, Jayaraman S, Clements LW, Scherer MA, Stefansic JD, Jarnagin WR et al (2013) Evolution of image-guided liver surgery: transition from open to laparoscopic procedures. J Gastrointest Surg 17(7):1274–1282PubMedPubMedCentral
8.
Buell JF, Cherqui D, Geller DA, Orourke N, Iannitti D, Dagher I et al (2009) The international position on laparoscopic liver surgery: the Louisville Statement, 2008. Ann Surg 250(5):825–830PubMed
9.
Cai X, Li Z, Zhang Y, Yu H, Liang X, Jin R et al (2014) Laparoscopic liver resection and the learning curve: a 14-year, single-center experience. Surg Endosc 28:1334–1341PubMed
10.
Cauchy F, Fuks D, Nomi T, Schwarz L, Barbier L, Dokmak S et al (2015) Risk factors and consequences of conversion in laparoscopic major liver resection. Br J Surg 102:785–795PubMed
11.
Azagury DE, Dua MM, Barrese JC, Henderson JM, Buchs NC, Ris F et al (2015) Image-guided surgery. Curr Probl Surg 52:476–520PubMed
12.
Pessaux P, Diana M, Soler L, Piardi T, Mutter D, Marescaux J (2014) Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbeck’s Arch Surg 400(3):381–385
13.
Miner RC (2017) Image-guided neurosurgery. J Med Imaging Radiat Sci 48(4):328–335PubMed
14.
Kang X, Azizian M, Wilson E, Wu K, Martin AD, Kane TD et al (2014) Stereoscopic augmented reality for laparoscopic surgery. Surg Endosc 28(7):2227–2235PubMed
15.
Reichard D, Bodenstedt S, Suwelack S, Mayer B, Preukschas A, Wagner M et al (2015) Intraoperative on-the-fly organ-mosaicking for laparoscopic surgery. J Med Imaging 2(4):045001
16.
Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201PubMed
17.
Soler L, Nicolau S, Pessaux P, Mutter D, Marescaux J (2014) Real-time 3D image reconstruction guidance in liver resection surgery. Hepatobiliary Surg Nutr 3(2):73–81PubMedPubMedCentral
18.
Hammill CW, Clements LW, Stefansic JD, Wolf RF, Hansen PD, Gerber DA (2014) Evaluation of a minimally invasive image-guided surgery system for hepatic ablation procedures. Surg Innov 21(4):419–426PubMed
19.
Kingham TP, Scherer MA, Neese BW, Clements LW, Stefansic JD, Jarnagin WR (2012) Image-guided liver surgery: intraoperative projection of computed tomography images utilizing tracked ultrasound. HPB (Oxford) 14(9):594–603
20.
Tummers QRJG, Verbeek FPR, Prevoo HAJM, Braat AE, Baeten CIM, Frangioni JV et al (2014) First experience on laparoscopic near-infrared fluorescence imaging of hepatic uveal melanoma metastases using indocyanine green. Surg Innov 22(1):20–25PubMedPubMedCentral
21.
Thompson S, Schneider C, Bosi M, Gurusamy K, Ourselin S, Davidson B et al (2018) In vivo estimation of target registration errors during augmented reality laparoscopic surgery. Int J Comput Assist Radiol Surg 13(6):865–874PubMedPubMedCentral
22.
23.
Peterhans M, vom Berg A, Dagon B, Inderbitzin D, Baur C, Candinas D et al (2011) A navigation system for open liver surgery: design, workflow and first clinical applications. Int J Med Robot 7(1):7–16PubMed
24.
Cash DM, Miga MI, Glasgow SC, Dawant BM, Clements LW, Cao Z et al (2007) Concepts and preliminary data toward the realization of image-guided liver surgery. J Gastrointest Surg 11(7):844–859PubMed
25.
Teatini A, Pelanis E, Aghayan DL, Kumar RP, Palomar R, Fretland ÅA et al (2019) The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery. Sci Rep 9(1):18687PubMedPubMedCentral
26.
Thompson S, Totz J, Song YY, Johnsen S, Stoyanov D, Gurusamy K et al (2015) Accuracy validation of an image guided laparoscopy system for liver resection. SPIE Proc 9415(7):941509
27.
Totz J, Thompson S, Stoyanov D, Gurusamy K, Davidson BR, Hawkes DJ et al (2014) Fast semi-dense surface reconstruction from stereoscopic video in laparoscopic surgery. IPCAI 8498:206–215
28.
Kleemann M, Deichmann S, Esnaashari H, Besirevic A, Shahin O, Bruch H-P et al (2012) Laparoscopic navigated liver resection: technical aspects and clinical practice in benign liver tumors. Case Rep Surg 2012:265918PubMedPubMedCentral
29.
Thompson S, Stoyanov D, Schneider C, Gurusamy K, Ourselin S, Davidson B et al (2016) Hand–eye calibration for rigid laparoscopes using an invariant point. Int J Comput Assist Radiol Surg 11(6):1071–1080PubMedPubMedCentral
30.
Stoyanov D, Scarzanella MV, Pratt P, Yang G-Z (2010) Real-time stereo reconstruction in robotically assisted minimally invasive surgery. Med Image Comput Comput Assist Interv 13(Pt 1):275–282PubMed
31.
Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334
32.
Thompson S, Stoyanov D, Schneider C, Gurusamy K, Ourselin S, Davidson B et al (2016) Hand–eye calibration for rigid laparoscopes using an invariant point. Int J Comput Assist Radiol Surg 11(6):1–10
33.
Dindo D, Demartines N, Clavien P-A (2004) Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg 240(2):205–213PubMedPubMedCentral
34.
Luo H, Yin D, Zhang S, Xiao D, He B, Meng F et al (2019) Augmented reality navigation for liver resection with a stereoscopic laparoscope. Comput Methods Progr Biomed 7:105099
35.
Gibson E, Robu MR, Thompson S, Edwards PE, Schneider C, Gurusamy K et al (2017) Deep residual networks for automatic segmentation of laparoscopic videos of the liver. In: Webster RJ, Fei B (eds) Prog Biomed Opt Imaging—Proc SPIE. International Society for Optics and Photonics, London
36.
Polignano FM, Quyn AJ, De Figueiredo RSM, Henderson NA, Kulli C, Tait IS (2008) Laparoscopic versus open liver segmentectomy: prospective, case-matched, intention-to-treat analysis of clinical outcomes and cost effectiveness. Surg Endosc Other Interv Tech 22(12):2564–2570
37.
Abu Hilal M, Di Fabio F, Syed S, Wiltshire R, Dimovska E, Turner D et al (2013) Assessment of the financial implications for laparoscopic liver surgery: A single-centre UK cost analysis for minor and major hepatectomy. Surg Endosc Other Interv Tech 27:2542–2550
38.
Okamoto T, Onda S, Yanaga K, Suzuki N, Hattori A (2015) Clinical application of navigation surgery using augmented reality in hepatobiliary pancreatic surgery. Surg Today 45(4):397–406PubMed
39.
Plantefève R, Peterlik I, Haouchine N, Cotin S (2016) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann Biomed Eng 44(1):139–153PubMed
40.
Haouchine N, Cotin S, Peterlik I, Dequidt J, Kerrien E, Berger M et al (2015) Impact of soft tissue heterogeneity on augmented reality for liver surgery. IEEE Trans Vis Comput Graph 21(5):584–597PubMed
41.
Haouchine N, Dequidt J, Peterlik I (2014) Towards an accurate tracking of liver tumors for augmented reality in robotic assisted surgery. Springer, Berlin
42.
Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, dos Santos T et al (2014) Physics-based shape matching for intraoperative image guidance. Med Phys 41(11):111901PubMed
43.
Tinguely P, Fusaglia M, Freedman J, Banz V, Weber S, Candinas D et al (2017) Laparoscopic image-based navigation for microwave ablation of liver tumors: a multi-center study. Surg Endosc 31(10):4315–4324PubMed
44.
Song Y, Totz J, Thompson S, Johnsen S, Barratt D, Schneider C et al (2015) Locally rigid, vessel-based registration for laparoscopic liver surgery. Int J Comput Assist Radiol Surg 10(12):1951–1961PubMedPubMedCentral
45.
Fu Y, Robu MR, Koo B, Schneider C, van Laarhoven S, Stoyanov D et al (2019) More unlabelled data or label more data? A study on semi-supervised laparoscopic image segmentation. Springer, Cham
46.
Hayashi Y, Misawa K, Oda M, Hawkes DJ, Mori K (2015) Clinical application of a surgical navigation system based on virtual laparoscopy in laparoscopic gastrectomy for gastric cancer. Int J Comput Assist Radiol Surg 11:827–836PubMed
47.
Yasuda J, Okamoto T, Onda S, Fujioka S, Yanaga K, Suzuki N et al (2019) Application of image-guided navigation system for laparoscopic hepatobiliary surgery. Asian J Endosc Surg 13:39–45PubMed
48.
Robu MR, Edwards P, Ramalhinho J, Thompson S, Davidson B, Hawkes D et al (2017) Intelligent viewpoint selection for efficient CT to video registration in laparoscopic liver surgery. Int J Comput Assist Radiol Surg 12(7):1079–1088PubMedPubMedCentral
49.
Chmarra MK, Hansen R, Hofstad EF, Våpenstad C, Marvik R, Langø T (2013) Development of laparoscopic ultrasound training system. Surg Endosc Other Interv Tech 27:S165
50.
Solberg OV, Langø T, Tangen GA, Mårvik R, Ystgaard B, Rethy A et al (2009) Navigated ultrasound in laparoscopic surgery. Minim Invasive Ther Allied Technol 18(1):36–53PubMed
51.
Modrzejewski R, Collins T, Seeliger B, Bartoli A, Hostettler A, Marescaux J (2019) An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions. Int J Comput Assist Radiol Surg 14:1237–1245PubMed
52.
53.
Beermann M, Lindeberg J, Engstrand J, Galmén K, Karlgren S, Stillström D et al (2019) 1000 consecutive ablation sessions in the era of computer assisted image guidance: lessons learned. Eur J Radiol open 6:1–8PubMed
54.
Buchs NC, Volonte F, Pugin F, Toso C, Fusaglia M, Gavaghan K et al (2013) Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery. J Surg Res 184(2):825–831PubMed
Metadata
Title
Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study
Authors
C. Schneider
S. Thompson
J. Totz
Y. Song
M. Allam
M. H. Sodergren
A. E. Desjardins
D. Barratt
S. Ourselin
K. Gurusamy
D. Stoyanov
M. J. Clarkson
D. J. Hawkes
B. R. Davidson
Publication date
01-10-2020
Publisher
Springer US
Published in
Surgical Endoscopy / Issue 10/2020
Print ISSN: 0930-2794
Electronic ISSN: 1432-2218
DOI
https://doi.org/10.1007/s00464-020-07807-x

Other articles of this Issue 10/2020

Surgical Endoscopy 10/2020 Go to the issue

OriginalPaper

Cost-effectiveness analysis of laparoscopic and open surgery in routine Swedish care for colorectal cancer

Dynamic Manuscript

Anterior approach for pure laparoscopic donor right hepatectomy

OriginalPaper

Safety and short-term effectiveness of endoscopic sleeve gastroplasty using overstitch: preliminary report from a multicenter study

OriginalPaper

Laparoscopic colorectal cancer surgery reduces the adverse impacts of sarcopenia on postoperative outcomes: a propensity score-matched analysis

2019 SAGES Oral

The contribution of specific enhanced recovery after surgery (ERAS) protocol elements to reduced length of hospital stay after ventral hernia repair

2019 SAGES Oral

Is obesity a contraindication for kidney donation?