Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 6/2018

Open Access 01-06-2018 | Original Article

Global rigid registration of CT to video in laparoscopic liver surgery

Authors: Maria R. Robu, João Ramalhinho, Stephen Thompson, Kurinchi Gurusamy, Brian Davidson, David Hawkes, Danail Stoyanov, Matthew J. Clarkson

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 6/2018

Login to get access

Abstract

Purpose

Image-guidance systems have the potential to aid in laparoscopic interventions by providing sub-surface structure information and tumour localisation. The registration of a preoperative 3D image with the intraoperative laparoscopic video feed is an important component of image guidance, which should be fast, robust and cause minimal disruption to the surgical procedure. Most methods for rigid and non-rigid registration require a good initial alignment. However, in most research systems for abdominal surgery, the user has to manually rotate and translate the models, which is usually difficult to perform quickly and intuitively.

Methods

We propose a fast, global method for the initial rigid alignment between a 3D mesh derived from a preoperative CT of the liver and a surface reconstruction of the intraoperative scene. We formulate the shape matching problem as a quadratic assignment problem which minimises the dissimilarity between feature descriptors while enforcing geometrical consistency between all the feature points. We incorporate a novel constraint based on the liver contours which deals specifically with the challenges introduced by laparoscopic data.

Results

We validate our proposed method on synthetic data, on a liver phantom and on retrospective clinical data acquired during a laparoscopic liver resection. We show robustness over reduced partial size and increasing levels of deformation. Our results on the phantom and on the real data show good initial alignment, which can successfully converge to the correct position using fine alignment techniques. Furthermore, since we can pre-process the CT scan before surgery, the proposed method runs faster than current algorithms.

Conclusion

The proposed shape matching method can provide a fast, global initial registration, which can be further refined by fine alignment methods. This approach will lead to a more usable and intuitive image-guidance system for laparoscopic liver surgery.
Literature
1.
go back to reference Bartoli A, Collins T, Bourdel N, Canis M (2012) Computer assisted minimally invasive surgery: Is medical computer vision the answer to improving laparosurgery? J Med Hypotheses 79(6):858–863CrossRef Bartoli A, Collins T, Bourdel N, Canis M (2012) Computer assisted minimally invasive surgery: Is medical computer vision the answer to improving laparosurgery? J Med Hypotheses 79(6):858–863CrossRef
2.
go back to reference Hughes-Hallett A, Pratt P, Dilley J, Vale J, Darzi A, Mayer E (2015) Augmented reality: 3d image-guided surgery. Cancer Imaging 15(1):O8CrossRefPubMedCentral Hughes-Hallett A, Pratt P, Dilley J, Vale J, Darzi A, Mayer E (2015) Augmented reality: 3d image-guided surgery. Cancer Imaging 15(1):O8CrossRefPubMedCentral
3.
go back to reference Plantefève R, Peterlik I, Haouchine N, Cotin S (2016) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann Biomed Eng 44(1):139–153CrossRefPubMed Plantefève R, Peterlik I, Haouchine N, Cotin S (2016) Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery. Ann Biomed Eng 44(1):139–153CrossRefPubMed
4.
go back to reference Rucker DC, Wu Y, Clements LW, Ondrake JE, Pheiffer TS, Simpson AL, Jarnagin WR, Miga MI (2014) A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imaging 33(1):147–158CrossRefPubMed Rucker DC, Wu Y, Clements LW, Ondrake JE, Pheiffer TS, Simpson AL, Jarnagin WR, Miga MI (2014) A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imaging 33(1):147–158CrossRefPubMed
5.
go back to reference Thompson S, Totz J, Song Y, Johnsen S, Stoyanov D, Ourselin S, Gurusamy K, Scheider C, Davidson B, Hawkes D, Clarkson MJ (2015) Accuracy validation of an image guided laparoscopy system for liver resection. In: SPIE, vol 9415, p 941509 Thompson S, Totz J, Song Y, Johnsen S, Stoyanov D, Ourselin S, Gurusamy K, Scheider C, Davidson B, Hawkes D, Clarkson MJ (2015) Accuracy validation of an image guided laparoscopy system for liver resection. In: SPIE, vol 9415, p 941509
6.
go back to reference Clements LW, Chapman WC, Dawant BM, Galloway RL, Miga MI (2008) Robust surface registration using salient anatomical features for image-guided liver surgery: algorithm and validation. J Med Phys 35(6):2528CrossRef Clements LW, Chapman WC, Dawant BM, Galloway RL, Miga MI (2008) Robust surface registration using salient anatomical features for image-guided liver surgery: algorithm and validation. J Med Phys 35(6):2528CrossRef
7.
go back to reference dos Santos TR, Seitel A, Kilgus T, Suwelack S, Wekerle AL, Kenngott H, Speidel S, Schlemmer HP, Meinzer HP, Heimann T, Maier-Hein L (2014) Pose-independent surface matching for intra-operative soft-tissue marker-less registration. J Med Image Anal 18(7):1101–1114CrossRef dos Santos TR, Seitel A, Kilgus T, Suwelack S, Wekerle AL, Kenngott H, Speidel S, Schlemmer HP, Meinzer HP, Heimann T, Maier-Hein L (2014) Pose-independent surface matching for intra-operative soft-tissue marker-less registration. J Med Image Anal 18(7):1101–1114CrossRef
8.
go back to reference Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, dos Santos T, Maier-Hein L, Wagner M, Wünscher J, Kenngott H, Müller BP, Speidel S (2014) Physics-based shape matching for intraoperative image guidance. J Med Phys 41(11):111901CrossRef Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, dos Santos T, Maier-Hein L, Wagner M, Wünscher J, Kenngott H, Müller BP, Speidel S (2014) Physics-based shape matching for intraoperative image guidance. J Med Phys 41(11):111901CrossRef
9.
go back to reference Maier-Hein L, Franz AM, dos Santos TR, Schmidt M, Fangerau M, Meinzer HP, Fitzpatrick JM (2012) Convergent iterative closest-point algorithm to accomodate anisotropic and inhomogenous localization error. IEEE Trans Pattern Anal Mach Intell 34(8):1520–1532CrossRefPubMed Maier-Hein L, Franz AM, dos Santos TR, Schmidt M, Fangerau M, Meinzer HP, Fitzpatrick JM (2012) Convergent iterative closest-point algorithm to accomodate anisotropic and inhomogenous localization error. IEEE Trans Pattern Anal Mach Intell 34(8):1520–1532CrossRefPubMed
10.
go back to reference Fitzgibbon AW (2003) Robust registration of 2d and 3d point sets. J Image Vis Comput 21(13):1145–1153CrossRef Fitzgibbon AW (2003) Robust registration of 2d and 3d point sets. J Image Vis Comput 21(13):1145–1153CrossRef
11.
go back to reference Cash DM, Miga MI, Glasgow SC, Dawant BM, Clements LW, Cao Z, Galloway RL, Chapman WC (2007) Concepts and preliminary data toward the realization of image-guided liver surgery. J Gastrointest Surg 11(7):844–859CrossRefPubMed Cash DM, Miga MI, Glasgow SC, Dawant BM, Clements LW, Cao Z, Galloway RL, Chapman WC (2007) Concepts and preliminary data toward the realization of image-guided liver surgery. J Gastrointest Surg 11(7):844–859CrossRefPubMed
12.
go back to reference Nicolau SA, Pennec X, Soler L, Buy X, Gangi A, Ayache N, Marescaux J (2009) An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. J Med Image Anal 13(3):494–506CrossRef Nicolau SA, Pennec X, Soler L, Buy X, Gangi A, Ayache N, Marescaux J (2009) An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. J Med Image Anal 13(3):494–506CrossRef
13.
go back to reference Teber D, Guven S, Simpfendörfer T, Baumhauer M, Güven EO, Yencilek F, Gözen AS, Rassweiler J (2009) Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur Urol 56(2):332–338CrossRefPubMed Teber D, Guven S, Simpfendörfer T, Baumhauer M, Güven EO, Yencilek F, Gözen AS, Rassweiler J (2009) Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur Urol 56(2):332–338CrossRefPubMed
14.
go back to reference Wild E, Teber D, Schmid D, Simpfendörfer T, Müller M, Baranski AC, Kenngott H, Kopka K, Maier-Hein L (2016) Robust augmented reality guidance with fluorescent markers in laparoscopic surgery. IJCARS 11(6):899–907 Wild E, Teber D, Schmid D, Simpfendörfer T, Müller M, Baranski AC, Kenngott H, Kopka K, Maier-Hein L (2016) Robust augmented reality guidance with fluorescent markers in laparoscopic surgery. IJCARS 11(6):899–907
15.
go back to reference Mountney P, Fallert J, Nicolau S, Soler L, Mewes PW (2014) An augmented reality framework for soft tissue surgery. MICCAI 17:423–431PubMed Mountney P, Fallert J, Nicolau S, Soler L, Mewes PW (2014) An augmented reality framework for soft tissue surgery. MICCAI 17:423–431PubMed
16.
go back to reference Feuerstein M, Mussack T, Heining SM, Navab N (2007) Registration-free laparoscope augmentation for intra-operative liver resection planning. In: Medical imaging 2007 Feuerstein M, Mussack T, Heining SM, Navab N (2007) Registration-free laparoscope augmentation for intra-operative liver resection planning. In: Medical imaging 2007
17.
go back to reference Fusaglia M, Hess H, Schwalbe M, Peterhans M, Tinguely P, Weber S, Lu H (2016) A clinically applicable laser-based image-guided system for laparoscopic liver procedures. IJCARS 11(8):1499–1513 Fusaglia M, Hess H, Schwalbe M, Peterhans M, Tinguely P, Weber S, Lu H (2016) A clinically applicable laser-based image-guided system for laparoscopic liver procedures. IJCARS 11(8):1499–1513
18.
go back to reference Raviv D, Dubrovina A, Kimmel R (2012) Hierarchical matching of non-rigid shapes. In: SSVM, pp 604–615 Raviv D, Dubrovina A, Kimmel R (2012) Hierarchical matching of non-rigid shapes. In: SSVM, pp 604–615
19.
go back to reference Leordeanu M, Hebert M (2005) A spectral technique for correspondence problems using pairwise constraints. ICCV 2:1482–1489 Leordeanu M, Hebert M (2005) A spectral technique for correspondence problems using pairwise constraints. ICCV 2:1482–1489
20.
go back to reference Huang QX, Adams B, Wicke M, Guibas LJ (2008) Non-rigid registration under isometric deformations. Comput Gr Forum 27(5):1449–1457CrossRef Huang QX, Adams B, Wicke M, Guibas LJ (2008) Non-rigid registration under isometric deformations. Comput Gr Forum 27(5):1449–1457CrossRef
21.
go back to reference Peyré G (2011) The numerical tours of signal processing. Comput Sci Eng 13(4):94–97CrossRef Peyré G (2011) The numerical tours of signal processing. Comput Sci Eng 13(4):94–97CrossRef
22.
go back to reference Gelfand N, Ikemoto L, Rusinkiewicz S, Levoy M (2003) Geometrically stable sampling for the ICP algorithm. In: 3DIM, pp 260–267 Gelfand N, Ikemoto L, Rusinkiewicz S, Levoy M (2003) Geometrically stable sampling for the ICP algorithm. In: 3DIM, pp 260–267
23.
go back to reference Yang J, Zhang Q, Xiao Y, Cao Z (2017) Toldi: an effective and robust approach for 3d local shape description. Pattern Recognit 65:175–187CrossRef Yang J, Zhang Q, Xiao Y, Cao Z (2017) Toldi: an effective and robust approach for 3d local shape description. Pattern Recognit 65:175–187CrossRef
24.
go back to reference Sethian JA (1996) A fast marching level set method for monotonically advancing fronts. PNAS 93(4):1591–1595 Sethian JA (1996) A fast marching level set method for monotonically advancing fronts. PNAS 93(4):1591–1595
26.
go back to reference Crane K, Weischedel C, Wardetzky M (2013) Geodesics in heat: a new approach to computing distance based on heat flow. ACM Trans Gr 32(5):152:1–152:11CrossRef Crane K, Weischedel C, Wardetzky M (2013) Geodesics in heat: a new approach to computing distance based on heat flow. ACM Trans Gr 32(5):152:1–152:11CrossRef
27.
go back to reference Fischler MA, Bolles RC (1981) Random sample consensus: paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395CrossRef Fischler MA, Bolles RC (1981) Random sample consensus: paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395CrossRef
29.
go back to reference Cignoni P, Callieri M, Corsini M, Dellepiane M, Ganovelli F, Ranzuglia G (2008) MeshLab: an open-source mesh processing tool. In: Eurographics Italian chapter conference Cignoni P, Callieri M, Corsini M, Dellepiane M, Ganovelli F, Ranzuglia G (2008) MeshLab: an open-source mesh processing tool. In: Eurographics Italian chapter conference
30.
go back to reference Gibson ED, Robu MR, Thompson SA, Edwards PE, Schneider C, Gurusamy K, Davidson BR, Hawkes DJ, Barrat DC, Clarkson MJ (2017) Deep residual networks for automatic segmentation of laparoscopic videos of the liver. In: SPIE, vol 10135, pp 101351M–101351M–6 Gibson ED, Robu MR, Thompson SA, Edwards PE, Schneider C, Gurusamy K, Davidson BR, Hawkes DJ, Barrat DC, Clarkson MJ (2017) Deep residual networks for automatic segmentation of laparoscopic videos of the liver. In: SPIE, vol 10135, pp 101351M–101351M–6
31.
go back to reference Totz J, Thompson S, Stoyanov D, Gurusamy K, Davidson B, Hawkes DJ, Clarkson MJ (2014) Fast semi-dense surface reconstruction from stereoscopic video in laparoscopic surgery. IPCAI 8498:206–215 Totz J, Thompson S, Stoyanov D, Gurusamy K, Davidson B, Hawkes DJ, Clarkson MJ (2014) Fast semi-dense surface reconstruction from stereoscopic video in laparoscopic surgery. IPCAI 8498:206–215
Metadata
Title
Global rigid registration of CT to video in laparoscopic liver surgery
Authors
Maria R. Robu
João Ramalhinho
Stephen Thompson
Kurinchi Gurusamy
Brian Davidson
David Hawkes
Danail Stoyanov
Matthew J. Clarkson
Publication date
01-06-2018
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 6/2018
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-018-1781-z

Other articles of this Issue 6/2018

International Journal of Computer Assisted Radiology and Surgery 6/2018 Go to the issue