Skip to main content
Top
Published in: BMC Medical Imaging 1/2020

Open Access 01-12-2020 | Magnetic Resonance Imaging | Research article

Utility of deep learning networks for the generation of artificial cardiac magnetic resonance images in congenital heart disease

Authors: Gerhard-Paul Diller, Julius Vahle, Robert Radke, Maria Luisa Benesch Vidal, Alicia Jeanette Fischer, Ulrike M. M. Bauer, Samir Sarikouch, Felix Berger, Philipp Beerbaum, Helmut Baumgartner, Stefan Orwat, for the German Competence Network for Congenital Heart Defects Investigators

Published in: BMC Medical Imaging | Issue 1/2020

Login to get access

Abstract

Background

Deep learning algorithms are increasingly used for automatic medical imaging analysis and cardiac chamber segmentation. Especially in congenital heart disease, obtaining a sufficient number of training images and data anonymity issues remain of concern.

Methods

Progressive generative adversarial networks (PG-GAN) were trained on cardiac magnetic resonance imaging (MRI) frames from a nationwide prospective study to generate synthetic MRI frames. These synthetic frames were subsequently used to train segmentation networks (U-Net) and the quality of the synthetic training images, as well as the performance of the segmentation network was compared to U-Net-based solutions trained entirely on patient data.

Results

Cardiac MRI data from 303 patients with Tetralogy of Fallot were used for PG-GAN training. Using this model, we generated 100,000 synthetic images with a resolution of 256 × 256 pixels in 4-chamber and 2-chamber views. All synthetic samples were classified as anatomically plausible by human observers. The segmentation performance of the U-Net trained on data from 42 separate patients was statistically significantly better compared to the PG-GAN based training in an external dataset of 50 patients, however, the actual difference in segmentation quality was negligible (< 1% in absolute terms for all models).

Conclusion

We demonstrate the utility of PG-GANs for generating large amounts of realistically looking cardiac MRI images even in rare cardiac conditions. The generated images are not subject to data anonymity and privacy concerns and can be shared freely between institutions. Training supervised deep learning segmentation networks on this synthetic data yielded similar results compared to direct training on original patient data.
Literature
1.
go back to reference Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56.CrossRef Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44–56.CrossRef
2.
go back to reference Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer. 2018;18(8):500–10.CrossRef Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer. 2018;18(8):500–10.CrossRef
3.
go back to reference Diller G-P, Babu-Narayan S, Li W, Radojevic J, Kempny A, Uebing A, et al. Utility of machine learning algorithms in assessing patients with a systemic right ventricle. Eur Heart J Cardiovasc Imaging. 2019;20(8):925–31.CrossRef Diller G-P, Babu-Narayan S, Li W, Radojevic J, Kempny A, Uebing A, et al. Utility of machine learning algorithms in assessing patients with a systemic right ventricle. Eur Heart J Cardiovasc Imaging. 2019;20(8):925–31.CrossRef
4.
go back to reference Bai W, Sinclair M, Tarroni G, et al. Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J Cardiovasc Magn Reson. 2018;20(1):65.CrossRef Bai W, Sinclair M, Tarroni G, et al. Automated cardiovascular magnetic resonance image analysis with fully convolutional networks. J Cardiovasc Magn Reson. 2018;20(1):65.CrossRef
5.
go back to reference Karras T, Aila T, Laine S, Lehtinen J. Progressive growing of GANs for improved quality, stability, and variation. 2017;arXiv:1710.10196. Karras T, Aila T, Laine S, Lehtinen J. Progressive growing of GANs for improved quality, stability, and variation. 2017;arXiv:1710.10196.
8.
go back to reference Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC. Improved training of wasserstein gans. In: Advances in neural information processing systems; 2017. p. 5767–77. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC. Improved training of wasserstein gans. In: Advances in neural information processing systems; 2017. p. 5767–77.
9.
go back to reference Diller G-P, Kempny A, Alonso-Gonzalez R, Swan L, Uebing A, Li W, et al. Survival prospects and circumstances of death in contemporary adult congenital heart disease patients under follow-up at a large tertiary Centre. Circulation. 2015;132(22):2118–25.CrossRef Diller G-P, Kempny A, Alonso-Gonzalez R, Swan L, Uebing A, Li W, et al. Survival prospects and circumstances of death in contemporary adult congenital heart disease patients under follow-up at a large tertiary Centre. Circulation. 2015;132(22):2118–25.CrossRef
10.
go back to reference Beerbaum P, Barth P, Kropf S, Sarikouch S, Kelter-Kloepping A, Franke D, et al. Cardiac function by MRI in congenital heart disease: impact of consensus training on interinstitutional variance. J Magn Reson Imaging. 2009;30(5):956–66.CrossRef Beerbaum P, Barth P, Kropf S, Sarikouch S, Kelter-Kloepping A, Franke D, et al. Cardiac function by MRI in congenital heart disease: impact of consensus training on interinstitutional variance. J Magn Reson Imaging. 2009;30(5):956–66.CrossRef
11.
go back to reference Sarikouch S, Koerperich H, Dubowy K-O, Boethig D, Boettler P, Mir TS, et al. Impact of gender and age on cardiovascular function late after repair of tetralogy of Fallot: percentiles based on cardiac magnetic resonance. Circ Cardiovasc Imaging. 2011;4(6):703–11.CrossRef Sarikouch S, Koerperich H, Dubowy K-O, Boethig D, Boettler P, Mir TS, et al. Impact of gender and age on cardiovascular function late after repair of tetralogy of Fallot: percentiles based on cardiac magnetic resonance. Circ Cardiovasc Imaging. 2011;4(6):703–11.CrossRef
12.
go back to reference Orwat S, Diller G-P, Kempny A, Radke R, Peters B, Kühne T, et al. Myocardial deformation parameters predict outcome in patients with repaired tetralogy of Fallot. Heart. 2016;102(3):209–15.CrossRef Orwat S, Diller G-P, Kempny A, Radke R, Peters B, Kühne T, et al. Myocardial deformation parameters predict outcome in patients with repaired tetralogy of Fallot. Heart. 2016;102(3):209–15.CrossRef
13.
go back to reference Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. Cham: Springer International Publishing; 2015. p. 234–41. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. Cham: Springer International Publishing; 2015. p. 234–41.
14.
go back to reference Zhang J, Gajjala S, Agrawal P, Tison GH, Hallock LA, Beussink-Nelson L, et al. Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy. Circulation. 2018;138(16):1623–35.CrossRef Zhang J, Gajjala S, Agrawal P, Tison GH, Hallock LA, Beussink-Nelson L, et al. Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy. Circulation. 2018;138(16):1623–35.CrossRef
15.
go back to reference Chen C, Qin C, Qiu H, Tarroni G, Duan J, Bai W, Rueckert D. Deep learning for cardiac image segmentation: a review. 2019. arXiv:191103723. arXiv preprint. Chen C, Qin C, Qiu H, Tarroni G, Duan J, Bai W, Rueckert D. Deep learning for cardiac image segmentation: a review. 2019. arXiv:191103723. arXiv preprint.
16.
go back to reference Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In: Advances in neural information processing systems. 2014;2014:2672–80. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In: Advances in neural information processing systems. 2014;2014:2672–80.
17.
go back to reference Zhao M, Liu X, Liu H, Wong KKL. Super-resolution of cardiac magnetic resonance images using Laplacian pyramid based on generative adversarial networks. Comput Med Imaging Graph. 2020;80:101698.CrossRef Zhao M, Liu X, Liu H, Wong KKL. Super-resolution of cardiac magnetic resonance images using Laplacian pyramid based on generative adversarial networks. Comput Med Imaging Graph. 2020;80:101698.CrossRef
18.
go back to reference Diller G-P, Lammers AE, Babu-Narayan S, Li W, Radke RM, Baumgartner H, et al. Denoising and artefact removal for transthoracic echocardiographic imaging in congenital heart disease: utility of diagnosis specific deep learning algorithms. Int J Cardiovasc Imaging. 2019;35(12):2189–96.CrossRef Diller G-P, Lammers AE, Babu-Narayan S, Li W, Radke RM, Baumgartner H, et al. Denoising and artefact removal for transthoracic echocardiographic imaging in congenital heart disease: utility of diagnosis specific deep learning algorithms. Int J Cardiovasc Imaging. 2019;35(12):2189–96.CrossRef
19.
go back to reference Jin C-B, Kim H, Liu M, Jung W, Joo S, Park E, et al. Deep CT to MR synthesis using paired and unpaired data. Sensors (Basel). 2019;19(10):2361.CrossRef Jin C-B, Kim H, Liu M, Jung W, Joo S, Park E, et al. Deep CT to MR synthesis using paired and unpaired data. Sensors (Basel). 2019;19(10):2361.CrossRef
20.
go back to reference Shin HC, Tenenholtz NA, Rogers JK, Schwarz CG, Senjem ML, Gunter JL, et al. Medical image synthesis for data augmentation and anonymization using generative adversarial networks. Cham: Springer International Publishing; 2018. p. 1–11. Shin HC, Tenenholtz NA, Rogers JK, Schwarz CG, Senjem ML, Gunter JL, et al. Medical image synthesis for data augmentation and anonymization using generative adversarial networks. Cham: Springer International Publishing; 2018. p. 1–11.
Metadata
Title
Utility of deep learning networks for the generation of artificial cardiac magnetic resonance images in congenital heart disease
Authors
Gerhard-Paul Diller
Julius Vahle
Robert Radke
Maria Luisa Benesch Vidal
Alicia Jeanette Fischer
Ulrike M. M. Bauer
Samir Sarikouch
Felix Berger
Philipp Beerbaum
Helmut Baumgartner
Stefan Orwat
for the German Competence Network for Congenital Heart Defects Investigators
Publication date
01-12-2020
Publisher
BioMed Central
Published in
BMC Medical Imaging / Issue 1/2020
Electronic ISSN: 1471-2342
DOI
https://doi.org/10.1186/s12880-020-00511-1

Other articles of this Issue 1/2020

BMC Medical Imaging 1/2020 Go to the issue