Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Nuclear Medicine and Molecular Imaging
Published in: Nuclear Medicine and Molecular Imaging 2/2018

01-04-2018 | Review

Deep Learning in Nuclear Medicine and Molecular Imaging: Current Perspectives and Future Directions

Author: Hongyoon Choi

Published in: Nuclear Medicine and Molecular Imaging | Issue 2/2018

Login to get access

Abstract

Recent advances in deep learning have impacted various scientific and industrial fields. Due to the rapid application of deep learning in biomedical data, molecular imaging has also started to adopt this technique. In this regard, it is expected that deep learning will potentially affect the roles of molecular imaging experts as well as clinical decision making. This review firstly offers a basic overview of deep learning particularly for image data analysis to give knowledge to nuclear medicine physicians and researchers. Because of the unique characteristics and distinctive aims of various types of molecular imaging, deep learning applications can be different from other fields. In this context, the review deals with current perspectives of deep learning in molecular imaging particularly in terms of development of biomarkers. Finally, future challenges of deep learning application for molecular imaging and future roles of experts in molecular imaging will be discussed.
Literature
1.
Weber GM, Mandl KD, Kohane IS. Finding the missing link for big biomedical data. JAMA. 2014;311:2479–80.PubMed
2.
Kuo M-H, Sahama T, Kushniruk AW, Borycki EM, Grunwell DK. Health big data analytics: current perspectives, challenges and potential solutions. Int J Big Data Intell. 2014;1:114–26.CrossRef
3.
Bengio Y. Learning deep architectures for AI. Foundations and trends® in. Mach Learn. 2009;2:1–127.CrossRef
4.
5.
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis. 2015;115:211–52.CrossRef
6.
Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst. 2012;25:1090–8.
7.
Alipanahi B, Delong A, Weirauch MT, Frey BJ. Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning. Nat Biotechnol. 2015;33:831–8.CrossRefPubMed
8.
9.
Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M et al. A survey on deep learning in medical image analysis. arXiv:170205747. 2017.
10.
Xu Y, Mo T, Feng Q, Zhong P, Lai M, Eric I et al., editors. Deep learning of feature representation with multiple instance learning for medical image analysis. Acoustics, Speech and Signal Processing (ICASSP), 2014 I.E. International Conference; 2014.
11.
Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, et al. Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging. 2016;35:1299–312.CrossRefPubMed
12.
Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402–10.CrossRefPubMed
13.
Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–8.CrossRefPubMed
14.
Mamoshina P, Vieira A, Putin E, Zhavoronkov A. Applications of deep learning in biomedicine. Mol Pharm. 2016;13:1445–54.CrossRefPubMed
15.
Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Cogn Model. 1988;5:1.
16.
Bengio Y, Lamblin P, Popovici D, Larochelle H. Greedy layer-wise training of deep networks. Adv Neural Inf Proces Syst. 2006;19:153–160.
17.
Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science. 2006;313:504–7.CrossRefPubMed
18.
Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15:1929–58.
19.
Ioffe S, Szegedy C, editors. Batch normalization: accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning; 2015.
20.
Nair V, Hinton GE, editors. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10); 2010.
21.
Oquab M, Bottou L, Laptev I, Sivic J, editors. Learning and transferring mid-level image representations using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2014.
22.
LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86:2278–324.CrossRef
23.
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al., editors. Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition; 2015.
24.
He K, Zhang X, Ren S, Sun J, editors. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016.
25.
Vinyals O, Toshev A, Bengio S, Erhan D, editors. Show and tell: a neural image caption generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015.
26.
Girshick R, editor. Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision; 2015.
27.
Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. Adv Neural Inf Proces Syst. 2015;28:91–9.
28.
Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. arXiv:160600915. 2016.
29.
Badrinarayanan V, Kendall A, Cipolla R. Segnet: a deep convolutional encoder-decoder architecture for image segmentation. arXiv:151100561. 2015.
30.
Elmore JG, Longton GM, Carney PA, Geller BM, Onega T, Tosteson AN, et al. Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA. 2015;313:1122–32.CrossRefPubMedPubMedCentral
31.
Keyes JW Jr. SUV: standard uptake or silly useless value? J Nucl Med. 1995;36:1836–9.PubMed
32.
Wang H, Zhou Z, Li Y, Chen Z, Lu P, Wang W, et al. Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18 F-FDG PET/CT images. EJNMMI Res. 2017;7:11.CrossRefPubMedPubMedCentral
33.
Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, et al. Brain tumor segmentation with deep neural networks. Med Image Anal. 2017;35:18–31.CrossRefPubMed
34.
Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, et al. Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage. 2015;108:214–24.CrossRefPubMedPubMedCentral
35.
Roth HR, Lu L, Farag A, Shin H-C, Liu J, Turkbey EB et al., editors. Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin: Springer; 2015.
36.
Choi H, Jin KH. Fast and robust segmentation of the striatum using deep convolutional neural networks. J Neurosci Methods. 2016;274:146–53.CrossRefPubMed
37.
Shen W, Zhou M, Yang F, Yang C, Tian J, editors. Multi-scale convolutional neural networks for lung nodule classification. International Conference on Information Processing in Medical Imaging. Berlin: Springer; 2015.
38.
Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel SJ, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging. 2016;35:1160–9.CrossRefPubMed
39.
Kleesiek J, Urban G, Hubert A, Schwarz D, Maier-Hein K, Bendszus M, et al. Deep MRI brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage. 2016;129:460–9.CrossRefPubMed
40.
de Brebisson A, Montana G, editors. Deep neural networks for anatomical brain segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops; 2015.
41.
Chen H, Dou Q, Yu L, Heng P-A. Voxresnet: deep voxelwise residual networks for volumetric brain segmentation. arXiv:160805895. 2016.
42.
Avendi M, Kheradvar A, Jafarkhani H. A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI. Med Image Anal. 2016;30:108–19.CrossRefPubMed
43.
Charbonnier J-P, Van Rikxoort EM, Setio AA, Schaefer-Prokop CM, van Ginneken B, Ciompi F. Improving airway segmentation in computed tomography using leak detection with convolutional networks. Med Image Anal. 2017;36:52–60.CrossRefPubMed
44.
Pereira S, Pinto A, Alves V, Silva CA. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging. 2016;35:1240–51.CrossRefPubMed
45.
46.
Trebeschi S, van Griethuysen JJ, Lambregts DM, Lahaye MJ, Parmer C, Bakers FC, et al. Deep learning for fully-automated localization and segmentation of rectal cancer on multiparametric MR. Sci Rep. 2017;7
47.
Han D, Yu J, Yu Y, Zhang G, Zhong X, Lu J, et al. Comparison of 18 F-fluorothymidine and 18 F-fluorodeoxyglucose PET/CT in delineating gross tumor volume by optimal threshold in patients with squamous cell carcinoma of thoracic esophagus. Int J Radiat Oncol Biol Phys. 2010;76:1235–41.CrossRefPubMed
48.
Tylski P, Stute S, Grotus N, Doyeux K, Hapdey S, Gardin I, et al. Comparative assessment of methods for estimating tumor volume and standardized uptake value in 18F-FDG PET. J Nucl Med. 2010;51:268–76.CrossRefPubMed
49.
Dong C, Loy CC, He K, Tang X. Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell. 2016;38:295–307.CrossRefPubMed
50.
Kim J, Kwon Lee J, Mu Lee K, editors. Accurate image super-resolution using very deep convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016.
51.
52.
Jiao J, Ourselin S. Fast PET reconstruction using Multi-scale Fully Convolutional Neural Networks. arXiv:170407244. 2017.
53.
Jin KH, McCann MT, Froustey E, Unser M. Deep convolutional neural network for inverse problems in imaging. IEEE Trans Image Process. 2017;26:4509–22.CrossRef
54.
Isola P, Zhu J-Y, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. arXiv:161107004. 2016.
55.
Nie D, Trullo R, Petitjean C, Ruan S, Shen D. Medical Image Synthesis with Context-Aware Generative Adversarial Networks. arXiv:161205362. 2016.
56.
Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Med Phys. 2017;44:1408–19.CrossRefPubMed
57.
Bezrukov I, Mantlik F, Schmidt H, Schölkopf B, Pichler BJ, editors. MR-based PET attenuation correction for PET/MR imaging. Seminars in nuclear medicine. Amsterdam: Elsevier; 2013.
58.
Suk H-I, Lee S-W, Shen D. Initiative AsDN. Latent feature representation with stacked auto-encoder for AD/MCI diagnosis. Brain Struct Funct. 2015;220:841–59.CrossRefPubMed
59.
Choi H, Jin KH. Predicting cognitive decline with deep learning of brain metabolism and amyloid imaging. arXiv:170406033. 2017.
60.
61.
Oakden-Rayner L, Carneiro G, Bessen T, Nascimento JC, Bradley AP, Palmer LJ. Precision radiology: predicting longevity using feature engineering and deep learning methods in a radiomics framework. Sci Rep. 2017;7:1648.CrossRefPubMedPubMedCentral
62.
Li Z, Wang Y, Yu J, Guo Y, Cao W. Deep learning based Radiomics (DLR) and its usage in noninvasive IDH1 prediction for low grade glioma. Sci Rep. 2017;7
63.
Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Cavalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014;5
64.
Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48:441–6.CrossRefPubMedPubMedCentral
65.
Choi H. Functional connectivity patterns of autism spectrum disorder identified by deep feature learning. arXiv:170707932. 2017.
66.
67.
68.
69.
Gal Y, Ghahramani Z, editors. Dropout as a Bayesian approximation: representing model uncertainty in deep learning. International Conference on Machine Learning; 2016.
70.
Van De Vijver MJ, He YD, Van't Veer LJ, Dai H, Hart AA, Voskuil DW, et al. A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med. 2002;347:1999–2009.CrossRefPubMed
71.
Carneiro G, Nascimento J, Bradley AP, editors. Unregistered multiview mammogram analysis with pre-trained deep learning models. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berllin: Springer; 2015.
72.
73.
Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. arXiv:170502315. 2017.
Metadata
Title
Deep Learning in Nuclear Medicine and Molecular Imaging: Current Perspectives and Future Directions
Author
Hongyoon Choi
Publication date
01-04-2018
Publisher
Springer Berlin Heidelberg
Published in
Nuclear Medicine and Molecular Imaging / Issue 2/2018
Print ISSN: 1869-3474
Electronic ISSN: 1869-3482
DOI
https://doi.org/10.1007/s13139-017-0504-7

Other articles of this Issue 2/2018

Nuclear Medicine and Molecular Imaging 2/2018 Go to the issue

Review

Radiomics as a Quantitative Imaging Biomarker: Practical Considerations and the Current Standpoint in Neuro-oncologic Studies

Interesting Image

Focal Increased Tc-99m MDP Uptake in the Nutrient Foramen of the Femoral Diaphysis on Bone SPECT/CT

Original Article

A Systematic Comparative Evaluation of 68Ga-Labeled RGD Peptides Conjugated with Different Chelators

Case Report

Dual Pathologies of Parathyroid Adenoma and Papillary Thyroid Cancer on Fluorocholine and Fluorodeoxyglucose PET/CT

Original Article

Impact of 68Ga-DOTA-Peptide PET/CT on the Management of Gastrointestinal Neuroendocrine Tumour (GI-NET): Malaysian National Referral Centre Experience

Original Article

The Predictive Values of Lesion Size, F-18 FDG Avidity and I-131 Avidity for the Clinical Outcome of I-131 Treatment in Patients with Metastatic Differentiated Thyroid Carcinoma Only in the Lung