Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 2/2020

01-02-2020 | Echocardiography | Original Article

Fusion of 3D real-time echocardiography and cine MRI using a saliency analysis

Authors: Angélica Atehortúa, Mireille Garreau, Antoine Simon, Erwan Donal, Mathieu Lederlin, Eduardo Romero

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 2/2020

Login to get access

Abstract

Purpose

This paper presents a novel 3D multimodal registration strategy to fuse 3D real-time echocardiography images with cardiac cine MRI images. This alignment is performed in a saliency space, which is designed to maximize similarity between the two imaging modalities. This fusion improves the quality of the available information.

Methods

The method performs in two steps: temporal and spatial registrations. A temporal alignment is firstly achieved by nonlinearly matching pairs of correspondences between the two modalities using a dynamic time warping. A temporal registration is then carried out by applying nonrigid transformations in a common saliency space where normalized cross correlation between temporal pairs of salient volumes is maximized.

Results

The alignment performance was evaluated with a set of 18 subjects, 3 with cardiomyopathies and 15 healthy, by computing the Dice score and Hausdorff distance with respect to manual delineations of the left ventricle cavity in both modalities. A Dice score and Hausdorff distance of \(0.86\pm 0.04\) and \(13.61\pm 3.86\,\hbox {mm}\), respectively, were obtained. In addition, the deformation field was estimated by quantifying its foldings, obtaining a 98% of regularity in the deformation field.

Conclusions

The 3D multimodal registration strategy presented is performed in a saliency space. Unlike state-of-the-art methods, the presented one takes advantage of the temporal information of the heart to construct this common space, ending up with two well-aligned modalities and regular deformation fields. This preliminary study was evaluated on heterogeneous data composed of two different datasets, healthy and pathological cases, showing similar performances in both cases. Future work will focus on testing the presented strategy in a larger dataset with a balanced number of classes.
Literature
1.
go back to reference Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282CrossRef Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282CrossRef
2.
go back to reference Atehortúa A, Garreau M, Romero E (2017) Fusion of 4D echocardiography and cine cardiac magnetic resonance volumes using a salient spatio-temporal analysis. In 13th International conference on medical information processing and analysis (Vol. 10572, p. 105721A). International Society for Optics and Photonics Atehortúa A, Garreau M, Romero E (2017) Fusion of 4D echocardiography and cine cardiac magnetic resonance volumes using a salient spatio-temporal analysis. In 13th International conference on medical information processing and analysis (Vol. 10572, p. 105721A). International Society for Optics and Photonics
3.
go back to reference Badesch DB, Champion HC, Sanchez MAG, Hoeper MM, Loyd JE, Manes A, McGoon M, Naeije R, Olschewski H, Oudiz RJ (2009) Diagnosis and assessment of pulmonary arterial hypertension. J Am Coll Cardiol 54(1 Supplement):S55–S66CrossRef Badesch DB, Champion HC, Sanchez MAG, Hoeper MM, Loyd JE, Manes A, McGoon M, Naeije R, Olschewski H, Oudiz RJ (2009) Diagnosis and assessment of pulmonary arterial hypertension. J Am Coll Cardiol 54(1 Supplement):S55–S66CrossRef
4.
go back to reference Berndt DJ, Clifford J (1994) Using dynamic time warping to find patterns in time series. In: KDD workshop. vol. 10, pp. 359–370. Seattle, WA Berndt DJ, Clifford J (1994) Using dynamic time warping to find patterns in time series. In: KDD workshop. vol. 10, pp. 359–370. Seattle, WA
5.
go back to reference Betancur J, Simon A, Langella B, Leclercq C, Hernández A, Garreau M (2016) Synchronization and registration of cine magnetic resonance and dynamic computed tomography images of the heart. IEEE J Biomed Health Inform 20(5):1369–1376CrossRef Betancur J, Simon A, Langella B, Leclercq C, Hernández A, Garreau M (2016) Synchronization and registration of cine magnetic resonance and dynamic computed tomography images of the heart. IEEE J Biomed Health Inform 20(5):1369–1376CrossRef
6.
go back to reference Burt P, Adelson E (1983) The Laplacian pyramid as a compact image code. IEEE Trans Commun 31(4):532–540CrossRef Burt P, Adelson E (1983) The Laplacian pyramid as a compact image code. IEEE Trans Commun 31(4):532–540CrossRef
7.
go back to reference Dahlem K, Michels G, Kobe C, Bunck AC, Ten Freyhaus H, Pfister R (2017) Diagnosis of cardiac transthyretin amyloidosis based on multimodality imaging. Clin Res Cardiol 106(6):471–473CrossRef Dahlem K, Michels G, Kobe C, Bunck AC, Ten Freyhaus H, Pfister R (2017) Diagnosis of cardiac transthyretin amyloidosis based on multimodality imaging. Clin Res Cardiol 106(6):471–473CrossRef
8.
go back to reference Freeman WT, Adelson EH (1991) The design and use of steerable filters. IEEE Trans Pattern Anal Mach Intell 9:891–906CrossRef Freeman WT, Adelson EH (1991) The design and use of steerable filters. IEEE Trans Pattern Anal Mach Intell 9:891–906CrossRef
9.
go back to reference Grewenig S, Zimmer S, Weickert J (2011) Rotationally invariant similarity measures for nonlocal image denoising. J Vis Commun Image Represent 22(2):117–130CrossRef Grewenig S, Zimmer S, Weickert J (2011) Rotationally invariant similarity measures for nonlocal image denoising. J Vis Commun Image Represent 22(2):117–130CrossRef
10.
go back to reference Haugaa KH, Basso C, Badano LP, Bucciarelli-Ducci C, Cardim N, Gaemperli O, Galderisi M, Habib G, Knuuti J, Lancellotti P (2017) Comprehensive multi-modality imaging approach in arrhythmogenic cardiomyopathyan expert consensus document of the european association of cardiovascular imaging. Eur Heart J Cardiovasc Imaging 18(3):237–253CrossRef Haugaa KH, Basso C, Badano LP, Bucciarelli-Ducci C, Cardim N, Gaemperli O, Galderisi M, Habib G, Knuuti J, Lancellotti P (2017) Comprehensive multi-modality imaging approach in arrhythmogenic cardiomyopathyan expert consensus document of the european association of cardiovascular imaging. Eur Heart J Cardiovasc Imaging 18(3):237–253CrossRef
11.
go back to reference Huang X, Hill NA, Ren J, Guiraudon G, Boughner D, Peters TM (2005) Dynamic 3d ultrasound and mr image registration of the beating heart. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 171–178. Springer Huang X, Hill NA, Ren J, Guiraudon G, Boughner D, Peters TM (2005) Dynamic 3d ultrasound and mr image registration of the beating heart. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 171–178. Springer
12.
go back to reference Itti L (2005) Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Vis Cognit 12(6):1093–1123CrossRef Itti L (2005) Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Vis Cognit 12(6):1093–1123CrossRef
13.
go back to reference James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fus 19:4–19CrossRef James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fus 19:4–19CrossRef
14.
go back to reference Kanai T, Kadoya N, Ito K, Onozato Y, Cho SY, Kishi K, Jingu K (2014) Evaluation of accuracy of B-spline transformation-based deformable image registration with different parameter settings for thoracic images. J Radiat Res 55(6):1163–1170CrossRef Kanai T, Kadoya N, Ito K, Onozato Y, Cho SY, Kishi K, Jingu K (2014) Evaluation of accuracy of B-spline transformation-based deformable image registration with different parameter settings for thoracic images. J Radiat Res 55(6):1163–1170CrossRef
15.
go back to reference Kiss G, Thorstensen A, Amundsen B, Claus P, D’hooge J, Torp H (2012) Fusion of 3d echocardiographic and cardiac magnetic resonance volumes. In: 2012 IEEE International Ultrasonics Symposium. pp. 126–129. IEEE Kiss G, Thorstensen A, Amundsen B, Claus P, D’hooge J, Torp H (2012) Fusion of 3d echocardiographic and cardiac magnetic resonance volumes. In: 2012 IEEE International Ultrasonics Symposium. pp. 126–129. IEEE
16.
go back to reference Klein S, Pluim JP, Staring M, Viergever MA (2009) Adaptive stochastic gradient descent optimisation for image registration. Int J Comput Vis 81(3):227CrossRef Klein S, Pluim JP, Staring M, Viergever MA (2009) Adaptive stochastic gradient descent optimisation for image registration. Int J Comput Vis 81(3):227CrossRef
17.
go back to reference Klein S, Staring M, Murphy K, Viergever MA, Pluim JPW (2010) Elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging 29(1):196–205CrossRef Klein S, Staring M, Murphy K, Viergever MA, Pluim JPW (2010) Elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging 29(1):196–205CrossRef
18.
go back to reference Klein S, Staring M, Pluim JP (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines. IEEE Trans Image Process 16(12):2879–2890CrossRef Klein S, Staring M, Pluim JP (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines. IEEE Trans Image Process 16(12):2879–2890CrossRef
19.
go back to reference Ma YL, Penney GP, Rinaldi CA, Cooklin M, Razavi R, Rhode KS (2009) Echocardiography to magnetic resonance image registration for use in image-guided cardiac catheterization procedures. Phys Med Biol 54(16):5039CrossRef Ma YL, Penney GP, Rinaldi CA, Cooklin M, Razavi R, Rhode KS (2009) Echocardiography to magnetic resonance image registration for use in image-guided cardiac catheterization procedures. Phys Med Biol 54(16):5039CrossRef
20.
go back to reference Mizuguchi Y, Oishi Y, Miyoshi H, Iuchi A, Nagase N, Oki T (2008) The functional role of longitudinal, circumferential, and radial myocardial deformation for regulating the early impairment of left ventricular contraction and relaxation in patients with cardiovascular risk factors: a study with two-dimensional strain imaging. J Am Soc Echocardiogr 21(10):1138–1144CrossRef Mizuguchi Y, Oishi Y, Miyoshi H, Iuchi A, Nagase N, Oki T (2008) The functional role of longitudinal, circumferential, and radial myocardial deformation for regulating the early impairment of left ventricular contraction and relaxation in patients with cardiovascular risk factors: a study with two-dimensional strain imaging. J Am Soc Echocardiogr 21(10):1138–1144CrossRef
21.
go back to reference Nagueh SF, Chang SM, Nabi F, Shah DJ, Estep JD (2017) Imaging to diagnose and manage patients in heart failure with reduced ejection fraction. Circ Cardiovasc Imaging 10(4):e005615CrossRef Nagueh SF, Chang SM, Nabi F, Shah DJ, Estep JD (2017) Imaging to diagnose and manage patients in heart failure with reduced ejection fraction. Circ Cardiovasc Imaging 10(4):e005615CrossRef
22.
go back to reference Olsen FJ, Bertelsen L, de Knegt MC, Christensen TE, Vejlstrup N, Svendsen JH, Jensen JS, Biering-Sorensen T (2016) Multimodality cardiac imaging for the assessment of left atrial function and the association with atrial arrhythmias. Circ Cardiovasc Imaging 9(10):e004947CrossRef Olsen FJ, Bertelsen L, de Knegt MC, Christensen TE, Vejlstrup N, Svendsen JH, Jensen JS, Biering-Sorensen T (2016) Multimodality cardiac imaging for the assessment of left atrial function and the association with atrial arrhythmias. Circ Cardiovasc Imaging 9(10):e004947CrossRef
23.
go back to reference Ou Y, Akbari H, Bilello M, Da X, Davatzikos C (2014) Comparative evaluation of registration algorithms in different brain databases with varying difficulty: results and insights. IEEE Trans Med Imaging 33(10):2039–2065CrossRef Ou Y, Akbari H, Bilello M, Da X, Davatzikos C (2014) Comparative evaluation of registration algorithms in different brain databases with varying difficulty: results and insights. IEEE Trans Med Imaging 33(10):2039–2065CrossRef
24.
go back to reference Perperidis D, Mohiaddin RH, Rueckert D (2005) Spatio-temporal free-form registration of cardiac MR image sequences. Med Image Anal 9(5):441–456CrossRef Perperidis D, Mohiaddin RH, Rueckert D (2005) Spatio-temporal free-form registration of cardiac MR image sequences. Med Image Anal 9(5):441–456CrossRef
25.
go back to reference Puyol-Anton E, Sinclair M, Gerber B, Amzulescu MS, Langet H, De Craene M, Aljabar P, Piro P, King AP (2017) A multimodal spatiotemporal cardiac motion atlas from MR and ultrasound data. Med Image Anal 40:96–110CrossRef Puyol-Anton E, Sinclair M, Gerber B, Amzulescu MS, Langet H, De Craene M, Aljabar P, Piro P, King AP (2017) A multimodal spatiotemporal cardiac motion atlas from MR and ultrasound data. Med Image Anal 40:96–110CrossRef
26.
go back to reference Rueckert D, Sonoda LI, Hayes C, Hill DL, Leach MO, Hawkes DJ (1999) Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans Med Imaging 18(8):712–721CrossRef Rueckert D, Sonoda LI, Hayes C, Hill DL, Leach MO, Hawkes DJ (1999) Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans Med Imaging 18(8):712–721CrossRef
27.
go back to reference Szasz O (1952) On products of summability methods. Proc Am Math Soc 3(2):257–263CrossRef Szasz O (1952) On products of summability methods. Proc Am Math Soc 3(2):257–263CrossRef
28.
go back to reference Thorpe S, Fize D, Marlot C (1996) Speed of processing in the human visual system. Nature 381(6582):520CrossRef Thorpe S, Fize D, Marlot C (1996) Speed of processing in the human visual system. Nature 381(6582):520CrossRef
29.
go back to reference Tobon-Gomez C, De Craene M, Mcleod K, Tautz L, Shi W, Hennemuth A, Prakosa A, Wang H, Carr-White G, Kapetanakis S (2013) Benchmarking framework for myocardial tracking and deformation algorithms: an open access database. Med Image Anal 17(6):632–648CrossRef Tobon-Gomez C, De Craene M, Mcleod K, Tautz L, Shi W, Hennemuth A, Prakosa A, Wang H, Carr-White G, Kapetanakis S (2013) Benchmarking framework for myocardial tracking and deformation algorithms: an open access database. Med Image Anal 17(6):632–648CrossRef
30.
go back to reference Valsangiacomo Buechel ER, Mertens LL (2012) Imaging the right heart: the use of integrated multimodality imaging. Eur Heart J 33(8):949–960CrossRef Valsangiacomo Buechel ER, Mertens LL (2012) Imaging the right heart: the use of integrated multimodality imaging. Eur Heart J 33(8):949–960CrossRef
31.
go back to reference Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, Gerig G (2006) User-guided 3d active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3):1116–1128CrossRef Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, Gerig G (2006) User-guided 3d active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3):1116–1128CrossRef
32.
go back to reference Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM international conference on Multimedia. pp. 815–824. ACM Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM international conference on Multimedia. pp. 815–824. ACM
33.
go back to reference Zhang W, Noble JA, Brady JM (2007) Spatio-temporal registration of real time 3d ultrasound to cardiovascular MR sequences. In: International conference on medical image computing and computer-assisted intervention. pp. 343–350. Springer Zhang W, Noble JA, Brady JM (2007) Spatio-temporal registration of real time 3d ultrasound to cardiovascular MR sequences. In: International conference on medical image computing and computer-assisted intervention. pp. 343–350. Springer
34.
go back to reference Zhang W, Noble JA, Brady JM (2007) Adaptive non-rigid registration of real time 3d ultrasound to cardiovascular MR images. In: Biennial international conference on information processing in medical imaging. pp. 50–61. Springer Zhang W, Noble JA, Brady JM (2007) Adaptive non-rigid registration of real time 3d ultrasound to cardiovascular MR images. In: Biennial international conference on information processing in medical imaging. pp. 50–61. Springer
35.
go back to reference Zhao N, Basarab A, Kouamé D, Tourneret JY (2016) Joint segmentation and deconvolution of ultrasound images using a hierarchical Bayesian model based on generalized Gaussian priors. IEEE Trans Image Process 25(8):3736–3750CrossRef Zhao N, Basarab A, Kouamé D, Tourneret JY (2016) Joint segmentation and deconvolution of ultrasound images using a hierarchical Bayesian model based on generalized Gaussian priors. IEEE Trans Image Process 25(8):3736–3750CrossRef
Metadata
Title
Fusion of 3D real-time echocardiography and cine MRI using a saliency analysis
Authors
Angélica Atehortúa
Mireille Garreau
Antoine Simon
Erwan Donal
Mathieu Lederlin
Eduardo Romero
Publication date
01-02-2020
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 2/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-019-02087-w

Other articles of this Issue 2/2020

International Journal of Computer Assisted Radiology and Surgery 2/2020 Go to the issue