Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

ACC 2024 Congress

Catch up on all the top stories and latest evidence in cardiology research with our ACC 2024 Congress hub page.
ADVANCED SEARCH
Log in
Top
Journal of Imaging Informatics in Medicine
Published in: Journal of Digital Imaging 4/2017

Open Access 01-08-2017

Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions

Authors: Zeynettin Akkus, Alfiia Galimzianova, Assaf Hoogi, Daniel L. Rubin, Bradley J. Erickson

Published in: Journal of Imaging Informatics in Medicine | Issue 4/2017

Login to get access

Abstract

Quantitative analysis of brain MRI is routine for many neurological diseases and conditions and relies on accurate segmentation of structures of interest. Deep learning-based segmentation approaches for brain MRI are gaining interest due to their self-learning and generalization ability over large amounts of data. As the deep learning architectures are becoming more mature, they gradually outperform previous state-of-the-art classical machine learning algorithms. This review aims to provide an overview of current deep learning-based segmentation approaches for quantitative brain MRI. First we review the current deep learning architectures used for segmentation of anatomical brain structures and brain lesions. Next, the performance, speed, and properties of deep learning approaches are summarized and discussed. Finally, we provide a critical assessment of the current state and identify likely future developments and trends.
Literature
1.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.CrossRefPubMed
2.
Lin D, Vasilakos AV, Tang Y, Yao Y: Neural networks for computer-aided diagnosis in medicine: A review. Neurocomputing 216:700–708, 2016CrossRef
3.
Kooi T et al.: Large scale deep learning for computer aided detection of mammographic lesions. Med. Image Anal. 35:303–312, 2017CrossRefPubMed
4.
Cheng J-Z et al.: Computer-aided diagnosis with deep learning architecture: Applications to breast lesions in US images and pulmonary nodules in CT scans. Sci. Rep. 6:24454, 2016CrossRefPubMedPubMedCentral
5.
6.
Y. LeCun et al., “Backpropagation applied to handwritten zip code recognition,” Neural Comput., vol. 1, no. 4, pp. 541–551, 1989.CrossRef
7.
Deng J, et al.: “ImageNet: A large-scale hierarchical image database,” in 2009 I.E. Conference on Computer Vision and Pattern Recognition, 2009.
8.
O. Russakovsky et al., “ImageNet large scale visual recognition challenge,” Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252, 2015.CrossRef
9.
Krizhevsky A, Sutskever I, Hinton GE: ImageNet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ Eds. Advances in neural information processing systems 25. USA: Curran Associates, Inc., 2012, pp. 1097–1105
10.
He K, Zhang X, Ren S, Sun J: “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification,” in 2015 I.E. International Conference on Computer Vision (ICCV), 2015.
11.
G. P. Mazzara, R. P. Velthuizen, J. L. Pearlman, H. M. Greenberg, and H. Wagner, “Brain tumor target volume determination for radiation treatment planning through automated MRI segmentation,” Int. J. Radiat. Oncol. Biol. Phys., vol. 59, no. 1, pp. 300–312, 2004.CrossRefPubMed
12.
S. K. Warfield, K. H. Zou, and W. M. Wells, “Simultaneous truth and performance level estimation (STAPLE): An algorithm for the validation of image segmentation,” IEEE Trans. Med. Imaging, vol. 23, no. 7, pp. 903–921, 2004.CrossRefPubMedPubMedCentral
13.
A. Akhondi-Asl, L. Hoyte, M. E. Lockhart, and S. K. Warfield, “A logarithmic opinion pool based STAPLE algorithm for the fusion of segmentations with associated reliability weights,” IEEE Trans. Med. Imaging, vol. 33, no. 10, pp. 1997–2009, 2014.CrossRefPubMedPubMedCentral
14.
A. Klein et al., “Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration,” Neuroimage, vol. 46, no. 3, pp. 786–802, 2009.CrossRefPubMedPubMedCentral
15.
S. M. Smith, “Fast robust automated brain extraction,” Hum. Brain Mapp., vol. 17, no. 3, pp. 143–155, 2002.CrossRefPubMed
16.
J. E. Iglesias, C.-Y. Liu, P. M. Thompson, and Z. Tu, “Robust brain extraction across datasets and comparison with publicly available methods,” IEEE Trans. Med. Imaging, vol. 30, no. 9, pp. 1617–1634, 2011.CrossRefPubMed
17.
J. Ashburner and K. J. Friston, “Unified segmentation,” Neuroimage, vol. 26, no. 3, pp. 839–851, 2005.CrossRefPubMed
18.
U. Vovk, F. Pernus, and B. Likar, “A review of methods for correction of intensity inhomogeneity in MRI,” IEEE Trans. Med. Imaging, vol. 26, no. 3, pp. 405–421, 2007.CrossRefPubMed
19.
L. G. Nyúl and J. K. Udupa, “On standardizing the MR image intensity scale,” Magn. Reson. Med., vol. 42, no. 6, pp. 1072–1081, 1999.CrossRefPubMed
20.
P. Coupe, P. Yger, S. Prima, P. Hellier, C. Kervrann, and C. Barillot, “An optimized blockwise nonlocal means denoising filter for 3-D magnetic resonance images,” IEEE Trans. Med. Imaging, vol. 27, no. 4, pp. 425–441, 2008.CrossRefPubMedPubMedCentral
21.
Kamnitsas K et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36:61–78, 2016CrossRefPubMed
22.
Pereira S, Pinto A, Alves V, Silva CA: “Brain Tumor Segmentation using Convolutional Neural Networks in MRI Images,” IEEE Trans. Med. Imaging, Mar. 2016.
23.
G. Wu, M. Kim, Q. Wang, Y. Gao, S. Liao, and D. Shen, “Unsupervised deep feature learning for deformable registration of MR brain images,” Med. Image Comput. Comput. Assist. Interv., vol. 16, no. Pt 2, pp. 649–656, 2013.PubMedPubMedCentral
24.
Kleesiek J et al.: Deep MRI brain extraction: A 3D convolutional neural network for skull stripping. Neuroimage 129:460–469, 2016CrossRefPubMed
25.
Gondara L: “Medical image denoising using convolutional denoising autoencoders,” arXiv [cs.CV], 2016.
26.
Havaei M et al.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35:18–31, 2016CrossRefPubMed
27.
28.
P. Moeskops et al., “Automatic segmentation of MR brain images with a convolutional neural network,” IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1252–1261, 2016.CrossRefPubMed
29.
Akkus Z, et al.: “Predicting 1p19q Chromosomal Deletion of Low-Grade Gliomas from MR Images using Deep Learning,” arXiv [cs.CV], 2016.
30.
Nie D, Dong N, Li W, Yaozong G, Dinggang S: “Fully convolutional networks for multi-modality isointense infant brain image segmentation,” in 2016 I.E. 13th International Symposium on Biomedical Imaging (ISBI), 2016.
31.
T. Brosch et al., “Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation,” IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1229–1239, 2016.CrossRefPubMed
32.
Q. Dou et al., “Automatic detection of cerebral Microbleeds from MR images via 3D convolutional neural networks,” IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1182–1195, 2016.CrossRef
33.
Srhoj-Egekher V, Manon JN, Viergever MA, Išgum I: “Automatic neonatal brain tissue segmentation with MRI,” in Medical Imaging 2013: Image Processing, 2013.
34.
35.
H. A. Vrooman et al., “Multi-spectral brain tissue segmentation using automatically trained k-nearest-neighbor classification,” Neuroimage, vol. 37, no. 1, pp. 71–81, 2007.CrossRefPubMed
36.
A. Makropoulos et al., “Automatic whole brain MRI segmentation of the developing neonatal brain,” IEEE Trans. Med. Imaging, vol. 33, no. 9, pp. 1818–1831, 2014.CrossRefPubMed
37.
Wang L et al.: LINKS: Learning-based multi-source IntegratioN frameworK for segmentation of infant brain images. Neuroimage 108:160–172, 2015CrossRefPubMed
38.
Moeskops P et al.: Automatic segmentation of MR brain images of preterm infants using supervised classification. Neuroimage 118:628–641, Sep. 2015CrossRefPubMed
39.
Chiţă SM, Benders M, Moeskops P, Kersbergen KJ, Viergever MA, Išgum I: “Automatic segmentation of the preterm neonatal brain with MRI using supervised classification,” in Medical Imaging 2013: Image Processing, 2013.
40.
A. de Brebisson, M. Giovanni: “Deep neural networks for anatomical brain segmentation,” in 2015 I.E. Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2015.
41.
Bao S, Siqi B, Chung ACS: “Multi-scale structured CNN with label consistency for brain MR image segmentation,” Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, pp. 1–5, 2016.
42.
D. W. Shattuck et al., “Construction of a 3D probabilistic atlas of human cortical structures,” Neuroimage, vol. 39, no. 3, pp. 1064–1080, 2008.CrossRefPubMed
43.
C. H. Sudre, M. J. Cardoso, W. H. Bouvy, G. J. Biessels, J. Barnes, and S. Ourselin, “Bayesian model selection for pathological neuroimaging data applied to white matter lesion segmentation,” IEEE Trans. Med. Imaging, vol. 34, no. 10, pp. 2079–2102, 2015.CrossRefPubMed
44.
A. Galimzianova, F. Pernuš, B. Likar, and Ž. Špiclin, “Stratified mixture modeling for segmentation of white-matter lesions in brain MR images,” Neuroimage, vol. 124, no. Pt A, pp. 1031–1043, 2016.CrossRefPubMed
45.
N. Weiss, D. Rueckert, and A. Rao, “Multiple sclerosis lesion segmentation using dictionary learning and sparse coding,” Med. Image Comput. Comput. Assist. Interv., vol. 16, no. Pt 1, pp. 735–742, 2013.PubMed
46.
Z. Karimaghaloo, H. Rivaz, D. L. Arnold, D. L. Collins, and T. Arbel, “Temporal hierarchical adaptive texture CRF for automatic detection of gadolinium-enhancing multiple sclerosis lesions in brain MRI,” IEEE Trans. Med. Imaging, vol. 34, no. 6, pp. 1227–1241, 2015.CrossRefPubMed
47.
X. Tomas-Fernandez and S. K. Warfield, “A model of population and subject (MOPS) intensities with application to multiple sclerosis lesion segmentation,” IEEE Trans. Med. Imaging, vol. 34, no. 6, pp. 1349–1361, 2015.CrossRefPubMedPubMedCentral
48.
N. Shiee, P.-L. Bazin, A. Ozturk, D. S. Reich, P. A. Calabresi, and D. L. Pham, “A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions,” Neuroimage, vol. 49, no. 2, pp. 1524–1535, 2010.CrossRefPubMed
49.
M. Prastawa, E. Bullitt, S. Ho, and G. Gerig, “A brain tumor segmentation framework based on outlier detection,” Med. Image Anal., vol. 8, no. 3, pp. 275–283, 2004.CrossRefPubMed
50.
S. Bauer, R. Wiest, L.-P. Nolte, and M. Reyes, “A survey of MRI-based medical image analysis for brain tumor studies,” Phys. Med. Biol., vol. 58, no. 13, pp. R97–129, 2013.CrossRefPubMed
51.
X. Lladó et al., “Automated detection of multiple sclerosis lesions in serial brain MRI,” Neuroradiology, vol. 54, no. 8, pp. 787–807, 2012.CrossRefPubMed
52.
D. García-Lorenzo, S. Francis, S. Narayanan, D. L. Arnold, and D. L. Collins, “Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging,” Med. Image Anal., vol. 17, no. 1, pp. 1–18, 2013.CrossRefPubMed
53.
Zhao L, Jia K: “Deep Feature Learning with Discrimination Mechanism for Brain Tumor Segmentation and Diagnosis,” in 2015 International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP), 2015.
54.
Dvořák P, Pavel D, Bjoern M: “Local Structure Prediction with Convolutional Neural Networks for Multimodal Brain Tumor Segmentation,” in Lecture Notes in Computer Science pp. 59–71, 2016.
55.
O. Maier, C. Schröder, N. D. Forkert, T. Martinetz, and H. Handels, “Classifiers for ischemic stroke lesion segmentation: A comparison study,” PLoS One, vol. 10, no. 12, p. e0145118, 2015.CrossRefPubMedPubMedCentral
56.
Havaei M, Guizard N, Larochelle H, Jodoin PM: “Deep Learning Trends for Focal Brain Pathology Segmentation in MRI,” in Lecture Notes in Computer Science pp. 125–148, 2016.
57.
B. H. Menze et al., “The multimodal brain tumor image segmentation benchmark (BRATS),” IEEE Trans. Med. Imaging, vol. 34, no. 10, pp. 1993–2024, 2015.CrossRefPubMed
58.
Cho J, Lee K, Shin E, Choy G, Do S: “How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?,” arXiv [cs.LG], 2015.
59.
Lekadir K, et al.: “A Convolutional Neural Network for Automatic Characterization of Plaque Composition in Carotid Ultrasound,” IEEE J Biomed Health Inform, 2016.
60.
Long J, Shelhamer E, Darrell T: “Fully convolutional networks for semantic segmentation,” in 2015 I.E. Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
61.
Yosinski J, Clune J, Bengio Y, Lipson H: How transferable are features in deep neural networks? In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ Eds. Advances in neural information processing systems 27. USA: Curran Associates, Inc., 2014, pp. 3320–3328
62.
H.-C. Shin et al., “Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning,” IEEE Trans. Med. Imaging, vol. 35, no. 5, pp. 1285–1298, 2016.CrossRefPubMed
63.
van Ginneken B, Setio AAA, Jacobs C, Ciompi F: “Off-the-shelf convolutional neural network features for pulmonary nodule detection in computed tomography scans,” in 2015 I.E. 12th International Symposium on Biomedical Imaging (ISBI), 2015.
Metadata
Title
Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions
Authors
Zeynettin Akkus
Alfiia Galimzianova
Assaf Hoogi
Daniel L. Rubin
Bradley J. Erickson
Publication date
01-08-2017
Publisher
Springer International Publishing
Published in
Journal of Imaging Informatics in Medicine / Issue 4/2017
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI
https://doi.org/10.1007/s10278-017-9983-4

Other articles of this Issue 4/2017

Journal of Digital Imaging 4/2017 Go to the issue

OriginalPaper

Medical Image Data and Datasets in the Era of Machine Learning—Whitepaper from the 2016 C-MIMI Meeting Dataset Session

OriginalPaper

Computerized Classification of Pneumoconiosis on Digital Chest Radiography Artificial Neural Network with Three Stages

OriginalPaper

Thyroid Nodule Classification in Ultrasound Images by Fine-Tuning Deep Convolutional Neural Network

OriginalPaper

Performance of an Artificial Multi-observer Deep Neural Network for Fully Automated Segmentation of Polycystic Kidneys

Editorial

Machine Learning: Discovering the Future of Medical Imaging

OriginalPaper

Pixel-Level Deep Segmentation: Artificial Intelligence Quantifies Muscle on Computed Tomography for Body Morphometric Analysis