Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 10/2018

01-10-2018 | Original Article

Preoperative liver registration for augmented monocular laparoscopy using backward–forward biomechanical simulation

Authors: Erol Özgür, Bongjin Koo, Bertrand Le Roy, Emmanuel Buc, Adrien Bartoli

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 10/2018

Login to get access

Abstract

Purpose

Augmented reality for monocular laparoscopy from a preoperative volume such as CT is achieved in two steps. The first step is to segment the organ in the preoperative volume and reconstruct its 3D model. The second step is to register the preoperative 3D model to an initial intraoperative laparoscopy image. To date, there does not exist an automatic initial registration method to solve the second step for the liver in the de facto operating room conditions of monocular laparoscopy. Existing methods attempt to solve for both deformation and pose simultaneously, leading to nonconvex problems with no optimal solution algorithms.

Methods

We propose in contrast to break the problem down into two parts, solving for (i) deformation and (ii) pose. Part (i) simulates biomechanical deformations from the preoperative to the intraoperative state to predict the liver’s unknown intraoperative shape by modeling gravity, the abdominopelvic cavity’s pressure and boundary conditions. Part (ii) rigidly registers the simulated shape to the laparoscopy image using contour cues.

Results

Our formulation leads to a well-posed problem, contrary to existing methods. This is because it exploits strong environment priors to complement the weak laparoscopic visual cues.

Conclusion

Quantitative results with in silico and phantom experiments and qualitative results with laparosurgery images for two patients show that our method outperforms the state-of-the-art in accuracy and registration time.
Literature
1.
go back to reference Wolf I, Vetter M, Wegner I, Böttger T, Nolden M, Schöbinger M, Hastenteufel M, Kunert T, Meinzer HP (2005) The medical imaging interaction toolkit. Med Image Anal 9(6):594–604CrossRefPubMed Wolf I, Vetter M, Wegner I, Böttger T, Nolden M, Schöbinger M, Hastenteufel M, Kunert T, Meinzer HP (2005) The medical imaging interaction toolkit. Med Image Anal 9(6):594–604CrossRefPubMed
2.
go back to reference Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201CrossRefPubMed Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201CrossRefPubMed
3.
go back to reference Bernhardt S, Nicolau SA, Soler L, Doignon C (2017) The status of augmented reality in laparoscopic surgery as of 2016. Med Imaging Anal 37:66–90CrossRef Bernhardt S, Nicolau SA, Soler L, Doignon C (2017) The status of augmented reality in laparoscopic surgery as of 2016. Med Imaging Anal 37:66–90CrossRef
4.
go back to reference Koo B, Ozgur E, Le Roy B, Buc E, Bartoli A (2017) Deformable registration of a preoperative 3D liver volume to a laparoscopy image using contour and shading cues. In: MICCAI Koo B, Ozgur E, Le Roy B, Buc E, Bartoli A (2017) Deformable registration of a preoperative 3D liver volume to a laparoscopy image using contour and shading cues. In: MICCAI
5.
go back to reference Adagolodjo Y, Trivisonne R, Haouchine N, Cotin S, Courtecuisse H (2017) Silhouette-based pose estimation for deformable organs application to surgical augmented reality. In: IROS Adagolodjo Y, Trivisonne R, Haouchine N, Cotin S, Courtecuisse H (2017) Silhouette-based pose estimation for deformable organs application to surgical augmented reality. In: IROS
6.
go back to reference Johnsen SF, Thompson S, Clarkson MJ, Modat M, Song Y, Totz J, Gurusamy K, Davidson B, Taylor ZA, Hawkes DJ, Ourselin S (2015) Database-based estimation of liver deformation under pneumoperitoneum for surgical image-guidance and simulation. In: MICCAI Johnsen SF, Thompson S, Clarkson MJ, Modat M, Song Y, Totz J, Gurusamy K, Davidson B, Taylor ZA, Hawkes DJ, Ourselin S (2015) Database-based estimation of liver deformation under pneumoperitoneum for surgical image-guidance and simulation. In: MICCAI
7.
go back to reference Bano J, Hostettler A, Nicolau SA, Doignon C, Wu HS, Huang MH, Soler L, Marescaux J (2012) Simulation of the abdominal wall and its arteries after pneumoperitoneum for guidance of port positioning in laparoscopic surgery. In: ISVC Bano J, Hostettler A, Nicolau SA, Doignon C, Wu HS, Huang MH, Soler L, Marescaux J (2012) Simulation of the abdominal wall and its arteries after pneumoperitoneum for guidance of port positioning in laparoscopic surgery. In: ISVC
8.
go back to reference Nimura Y, Qu JD, Hayashi Y, Oda M, Kitasaka T, Hashizume M, Misawa K, Mori K (2015) Pneumoperitoneum simulation based on mass-spring-damper models for laparoscopic surgical planning. J Med Imaging 2(4):044004CrossRef Nimura Y, Qu JD, Hayashi Y, Oda M, Kitasaka T, Hashizume M, Misawa K, Mori K (2015) Pneumoperitoneum simulation based on mass-spring-damper models for laparoscopic surgical planning. J Med Imaging 2(4):044004CrossRef
9.
go back to reference Bano J, Hostettler A, Nicolau SA, Cotin S, Doignon C, Wu HS, Huang MH, Soler L, Marescaux J (2012) Simulation of pneumoperitoneum for laparoscopic surgery planning. In: MICCAI Bano J, Hostettler A, Nicolau SA, Cotin S, Doignon C, Wu HS, Huang MH, Soler L, Marescaux J (2012) Simulation of pneumoperitoneum for laparoscopic surgery planning. In: MICCAI
10.
go back to reference Collins T, Pizarro D, Bartoli A, Bourdel N, Canis M (2014) Computer-aided laparoscopic myomectomy by augmenting the uterus with pre-operative MRI data. In: ISMAR Collins T, Pizarro D, Bartoli A, Bourdel N, Canis M (2014) Computer-aided laparoscopic myomectomy by augmenting the uterus with pre-operative MRI data. In: ISMAR
11.
go back to reference Bernhardt S, Nicolau S, Bartoli A, Agnus V, Soler L, Doignon C (2015) Using shading to register an intraoperative CT scan to a laparoscopic image. Workshop CARE at MICCAI Bernhardt S, Nicolau S, Bartoli A, Agnus V, Soler L, Doignon C (2015) Using shading to register an intraoperative CT scan to a laparoscopic image. Workshop CARE at MICCAI
12.
go back to reference Saito A, Nakao M, Uranishi Y, Matsuda T (2015) Deformation estimation of elastic bodies using multiple silhouette images for endoscopic image augmentation. In: ISMAR Saito A, Nakao M, Uranishi Y, Matsuda T (2015) Deformation estimation of elastic bodies using multiple silhouette images for endoscopic image augmentation. In: ISMAR
13.
go back to reference Haouchine N, Roy F, Untereiner L, Cotin S (2016) Using contours as boundary conditions for elastic registration during minimally invasive hepatic surgery. In: IROS Haouchine N, Roy F, Untereiner L, Cotin S (2016) Using contours as boundary conditions for elastic registration during minimally invasive hepatic surgery. In: IROS
14.
go back to reference Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, Thiago S, Maier-Hein L, Wagner M, Wünscher J, Kenngott H, Müller BP, Spiedel S (2014) Physics-based shape matching for intraoperative image guidance. Med Phys 41(11):111901CrossRefPubMed Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, Thiago S, Maier-Hein L, Wagner M, Wünscher J, Kenngott H, Müller BP, Spiedel S (2014) Physics-based shape matching for intraoperative image guidance. Med Phys 41(11):111901CrossRefPubMed
15.
go back to reference Bender J, Koschier D, Charrier P, Weber D (2014) Position-based simulation of continuous materials. Comput Graph 44:1–10CrossRef Bender J, Koschier D, Charrier P, Weber D (2014) Position-based simulation of continuous materials. Comput Graph 44:1–10CrossRef
16.
go back to reference Toledo M, Ribeiro PC (2009) Radiological evaluation of a liver simulator in comparison to a human real liver. In: International Nuclear Atlantic Conference, INAC Toledo M, Ribeiro PC (2009) Radiological evaluation of a liver simulator in comparison to a human real liver. In: International Nuclear Atlantic Conference, INAC
17.
go back to reference Mourcou Q, Fleury A, Franco C, Klopcic F, Vuillerme N (2015) Performance evaluation of smartphone inertial sensors measurement for range of motion. Sensors 15:23168–23187CrossRefPubMed Mourcou Q, Fleury A, Franco C, Klopcic F, Vuillerme N (2015) Performance evaluation of smartphone inertial sensors measurement for range of motion. Sensors 15:23168–23187CrossRefPubMed
18.
go back to reference Whiteley J (2005) The solution of inverse non-linear elasticity problems that arise when locating breast tumours. J Theor Med 6(3):143–149CrossRef Whiteley J (2005) The solution of inverse non-linear elasticity problems that arise when locating breast tumours. J Theor Med 6(3):143–149CrossRef
19.
go back to reference Carter T, Tanner C, Beechey-Newman N, Barratt D, Hawkes D (2008) MR navigated breast surgery: method and initial clinical experience. In: MICCAI Carter T, Tanner C, Beechey-Newman N, Barratt D, Hawkes D (2008) MR navigated breast surgery: method and initial clinical experience. In: MICCAI
20.
go back to reference Eiben B, Han L, Hipwell J, Mertzanidou T, Kabus S, Buelow T, Lorenz C, Newstead GM, Abe H, Keshtgar M, Ourselin S, Hawkes DJ (2013) Biomechanically guided prone-to-supine image registration of breast MRI using an estimated reference state. In: ISBI Eiben B, Han L, Hipwell J, Mertzanidou T, Kabus S, Buelow T, Lorenz C, Newstead GM, Abe H, Keshtgar M, Ourselin S, Hawkes DJ (2013) Biomechanically guided prone-to-supine image registration of breast MRI using an estimated reference state. In: ISBI
21.
go back to reference Sellier M (2011) An iterative method for the inverse elasto-static problem. J Fluids Struct 27:1461–1470CrossRef Sellier M (2011) An iterative method for the inverse elasto-static problem. J Fluids Struct 27:1461–1470CrossRef
22.
go back to reference Morin F, Courtecuisse H, Chabanas M, Payan Y (2015) Rest shape computation for highly deformable model of brain. Comput Methods Biomech Biomed Eng 18:2006–2007CrossRef Morin F, Courtecuisse H, Chabanas M, Payan Y (2015) Rest shape computation for highly deformable model of brain. Comput Methods Biomech Biomed Eng 18:2006–2007CrossRef
23.
go back to reference Eiben B, Vavourakis V, Hipwell J, Kabus S, Lorenz C, Buelow T, Hawkes DJ (2014) Breast deformation modelling: comparison of methods to obtain a patient specific unloaded configuration. In: Conference on medical imaging—image-guided procedures, robotic interventions, and modeling Eiben B, Vavourakis V, Hipwell J, Kabus S, Lorenz C, Buelow T, Hawkes DJ (2014) Breast deformation modelling: comparison of methods to obtain a patient specific unloaded configuration. In: Conference on medical imaging—image-guided procedures, robotic interventions, and modeling
24.
go back to reference Mackling M, Müller M, Chentanez N (2016) XPBD: position-based simulation of compliant constrained dynamics. In: MIG Mackling M, Müller M, Chentanez N (2016) XPBD: position-based simulation of compliant constrained dynamics. In: MIG
25.
go back to reference Lepetit V, Moreno-Noguer F, Fua P (2008) EPnP: an accurate O(n) solution to the PnP problem. Int J Comput Vis 81(2):155–166CrossRef Lepetit V, Moreno-Noguer F, Fua P (2008) EPnP: an accurate O(n) solution to the PnP problem. Int J Comput Vis 81(2):155–166CrossRef
26.
go back to reference Bosman J, Haouchine N, Dequidt J, Peterlik I (2014) The role of ligaments: patient-specific or scenario-specific? In: ISBMS Bosman J, Haouchine N, Dequidt J, Peterlik I (2014) The role of ligaments: patient-specific or scenario-specific? In: ISBMS
Metadata
Title
Preoperative liver registration for augmented monocular laparoscopy using backward–forward biomechanical simulation
Authors
Erol Özgür
Bongjin Koo
Bertrand Le Roy
Emmanuel Buc
Adrien Bartoli
Publication date
01-10-2018
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 10/2018
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-018-1842-3

Other articles of this Issue 10/2018

International Journal of Computer Assisted Radiology and Surgery 10/2018 Go to the issue