Deep Learning Methods to Process fMRI Data and Their Application in the Diagnosis of Cognitive Impairment: A Brief Overview and Our Opinion

Introduction

In recent years, brain imaging technology has become a hot topic within the field of neuroscience. Such technology includes functional Magnetic Resonance Imaging (fMRI) (Liu et al., 2013; Wang et al., 2018), electroencephalography (EEG) (Zhang et al., 2018), Magnetoencephalography (MEG) (Alotaiby et al., 2017), and functional Near Infrared Spectroscopy (fNIRS) (Vergotte et al., 2017). In particular, because of its space resolving power, fMRI, combined with deep learning methods, has been widely used in the study of neuroscience. For example, Radial Basis Function Networks (RBFN) were used to identify Autism Spectrum Disorder (ASD) (Vigneshwaran et al., 2015), Convolutional Neural Networks (CNN) were used to diagnose amnestic Mild Cognitive Impairment (aMCI) (Meszlényi et al., 2017) and Feedforward Neural Networks (FNN) were used to classify Schizophrenia (Patel et al., 2016). When we studied this topic, we mainly searched the literature from the database of Web of Science, and reviewed references to the application of deep learning methods in the field of fMRI from 2004 to the present. This paper first summarized the analyses of the applications, advantages and disadvantages of a variety of deep learning methods in processing fMRI data, and the names, target problem, cohort size, and accuracy of these methods were presented in Table 1. Then, considering the importance of studying cognitive disorders within the field of neuroscience, this paper reviewed the trend in the development of deep learning methods to process fMRI data for the diagnosis of cognitive disorders. Lastly, recommendations for future work in developing deep learning methods to process fMRI data for the diagnosis of cognitive disorders were stated. It is expected that the analysis may provide useful suggestions to the field of the deep learning methods of fMRI data with cognitive impairment.

Table 1. The names, target problem, cohort size, and accuracy of deep learning methods to process fMRI data.

Deep Learning Methods in fMRI Data Analysis

CNN

In recent years, CNN has drawn much attention and has been widely used in fMRI data processing, because valid features can be automatically extracted (Sarraf and Tofighi, 2016; Horikawa and Kamitani, 2017a; Meszlényi et al., 2017). For example, CNN was used to effectively classify fMRI data (Horikawa and Kamitani, 2017a) and to classify and diagnose Alzheimer's disease (AD) (Sarraf and Tofighi, 2016) and aMCI (Meszlényi et al., 2017). The CNN method consists of the following three applications: feature extraction, construction of auto-encoder and construction of 3-Dimensional Convolutional Neural Networks (3D-CNN) method.

Feature Extraction

To predict the lifetime of patients with brain tumors, CNN was used to extract features from fMRI data with the Support Vector Machine used for classification (Nie et al., 2016). Indeed, the study can use CNN to extract valid features, but whether the resulting features are redundant or not remains to be tested. In addition, the extracted features correspond to what features of the brain have yet to be explored.

Auto-Encoder

Feature maps can be obtained layer by layer using an auto-encoder (Huang et al., 2017). Auto-decoders had also drawn attention when interpreting fMRI data related to vision (Wen et al., 2017). In this study, unlike in the previous studies, Deconvolutional Neural Network was constructed to achieve the operation approximate to inverse convolution. However, for this method, it is still unknown for this method to explain all the region of vision. Generally, CNN is used mainly for two-dimensional data to build an auto-encoder, but there was a 3D-Convolutional Auto Encoder built for recognition of mild Traumatic Brain Injury (mTBI) (Zhao et al., 2017a), despite more samples being needed to validate the results of the study. In order to comprehend the changes in brain state following memory words, a sparse encoder was constructed and used in conjunction with CNN for classification (Firat et al., 2015).

3D-CNN

The 3D-CNN can also be used to process fMRI data. The data from Human Connectome project (HCP) was utilized to reconstruct the brain network with a higher accuracy (Zhao et al., 2017b). The methods of combining fMRI with structural magnetic resonance imaging had also been used for the classification of Attention deficit hyperactivity disorder (ADHD) (Zou et al., 2017). Although CNN can share the filter in the convolving layer and the number of parameters can be reduced in the pooling layer, the large amount of data required by the 3D network still demands extended training time.

FNN

FNN has been applied to analyze human brain changes in a simulated war environment (Floren et al., 2015). However, the FNN method is inefficient in training based on fMRI data due to the large input eigenvector. To reduce the dimension of the data and solve this problem, the fMRI data could be passed to the input layer through the encoder (Hu et al., 2016; Kim et al., 2016). However, the method proposed by Kim et al. is not suitable for the analysis of fMRI data on patients with different diseases. Thus, its generalization on different diseases needs to be strengthened. Based on the original encoder, a Deep Auto Encoder was used to classify fMRI data with MCI (Suk et al., 2016). But this study did not consider the dynamic relationship between different regions of the brain. In another approach, four hidden layers were added in between the encoder and the decoder in the FNN method (Patel et al., 2016). Apart from adding hidden layers, multiple encoders can also be stacked. For example, when diagnosing ASD, two denoising self-encoder were constructed (Heinsfeld et al., 2018). Multiple sparse auto-encoders had also been used (Guo et al., 2017). Yet, further experiments will need to be conducted with patients of different ages to validate the clinical value of these methods.

RBFN is a FNN, which uses radial basis functions in the hidden layer to non-linearly transform the input space. RBFN had been used to study HIV Associated Neurocognitive Disorder (HAND) (Abidin et al., 2016). The RBFN could be used to extract features from fMRI data (Vigneshwaran et al., 2015). This method can be extended to studies on women of different ages. Besides the basic feature selection, a feature selection method combining region of interest and voxel was designed to determine brain states under the different stimuli (Zafar et al., 2017). With regards to cognitive diseases, RBFN had also been used to classify different regions of the brain network impaired by aMCI (Jin et al., 2014). However, the sample size in this study is limited and so the impact of larger sample sizes needs to be investigated. Meanwhile, more careful regularization methods need to be developed to address the issue of over-fitting.

Other Deep Learning Methods

In addition to the widely used methods described above, Restricted Boltzmann Machine (RBM) and Deep Boltzmann Machine (DBM) are also used for fMRI data analysis. The performance of the RBM had been shown to be better than independent component analysis (Huang et al., 2016), but the functional relationship between different regions of the brain was not examined. The DBM was proposed to identify areas of the brain that were activated while the subject was watching videos (Han et al., 2015). In the future, emotion-oriented data may be considered for classification. In general, the greater the number of training samples available to the deep learning method is, the better the classification performance of the method will be. Recently, three data sets (OHSU, NYU, and Neuro) used in the diagnosis of ADHD were classified with an accuracy of 80.88, 37.41, and 44.4%, respectively (Kuang and He, 2014), and the quantitative difference indicates that the generalization ability of this method is limited. When the amount of data is insufficient, the synthesis of analog data can be considered. Avesani et al. used Liquid State Machine to train and classify both simulated and real data (Avesani et al., 2015), but the results of the simulation data do not reflect the true performance of the classification method.

There are also two time-dependent deep learning methods for fMRI data analysis. The first method, Spiking Neuron Networks, converted fMRI data of the brain during reading into pulse signals (Kasabov et al., 2017). This method performed well in analyzing the temporal characteristics of the data, but failed to analyse the spatial characteristics. The second method, Long Short Term Memory (LSTM), was used for the study of ASD. In this method, the training samples were obtained from the average time series extracted from fMRI data after preprocessing (Dvornek et al., 2017). The extraction of the average time series in this study affects the accuracy of the method classification.

To reduce the number of features extracted, a deep belief network (DBN) was used to initialize the parameters and sparse pre-training was performed on weight (Jang et al., 2017). This method reduced the number of parameters and decreased the degree of over-fitting, but it takes more time for calculations.

The Hopfield Neural Network was used many years ago to classify fMRI data on subjects touching different objects (Dixit and Mosier, 2004). However, due to the small amount of data collected, the performance of the method still needs to be fully verified.

The main problem currently facing the development of deep learning methods for fMRI data analysis is that the training methods used a smaller amount of data. Therefore, the studies of small sample may lead to obvious over-fitting problems. The training methods may take more time when number of sample increase, and more training time makes it more difficult to conduct in-depth and meticulous parameter adjustment work, resulting in hindered improvement of method performance.

Development of Deep Learning Methods for fMRI Data Analysis in Cognitive Impairment

Cognitive disorders, which is a general category including conditions such as Alzheimer's disease, mild cognitive impairment, and subjective cognitive decline, usually encompass the disorder of memory, language, visual space, execution, calculation, and understanding, and involve multiple regions of the brain and the associated abnormalities between these brain regions. This makes the diagnosis of cognitive disorders complicated. Therefore, the diagnosis of cognitive disorders has also become a very important issue in the field of neuroscience. In the following sections, the literature analyzing deep learning methods for classification of fMRI data in cognitive impairment studies will be reviewed. Recommendations and our opinion for the development of these methods are presented.

The Measure to Improve the Performance of Deep Learning Method in Cognitive Impairment

Firstly, for improving the model itself, as the number of fMRI data samples related to cognitive disorders is insufficient, we recommend to use a more refined regularization method, which includes the other hyperparameters of the deep learning methods. Secondly, the time cost of deep learning method training is higher, as the learning rate has an impact on the time taken for method training, the learning rate can be automatically adjusted according to the learning progress in future work. Thirdly, as the current method is aimed at studying a specific stage of the disease, a deep learning method to study all stages of the disease can be explored in the future.

From another perspective, we can apply methods for studying other diseases to cognitive impairment. In other words, deep learning methods that perform well in studying other diseases can be applied to analyzing fMRI data related to cognitive disorders. For example, the existing CNN method could be used to analyze fMRI data while the subject was watching videos (Güçlü and Van Gerven, 2017). As the method does not have to restart training, the training time is also greatly reduced.

Exploring Deep Learning Methods Specifically for fMRI Data Analysis of Cognitive Impairment

The functioning of the brain has now been shown to involve interactions and connection among multiple functional areas, and is a phenomenon also known as brain function network dynamics (Kim et al., 2017), in which the synchronous and coupling activity will occur between or among different areas of the brain when the brain performs a task. Cognitive disorders are due to variations in the dynamic relationships between pairs of specific brain regions and the network dynamics of multiple brain regions. Therefore, as current deep learning methods disregard dynamic considerations, when designing a deep learning method for fMRI data classification of cognitive impairment, it is suggested that the training of deep learning methods could be performed after calculating the coupling or synchronizing intensity between specific brain regions.

Conclusion

In conclusion, this study reviewed the recent literature of the deep learning used in fMRI data. Although these studies obtained promising results, more work needs to be done before conclusions can be confidently drawn in the field of deep learning used in fMRI. It must be noted that there are some limitations that occur both for deep learning and classical machine learning methods, even if in different proportions. For example, a small sample size is a source of overfitting in both cases, even if the number of samples required for training a classical machine learning method is extremely lower than the one needed for deep learning techniques (Salvatore et al., 2016). In addition, brain dynamics have not been taken seriously in the two ways. Compared with the classical machine learning methods, deep learning methods may have poorer performance under the condition of small sample size and training time cost (Nie et al., 2016), but we can make full use of the automatically-extracted features to improve accuracy of deep learning methods. In the future, we suggest collecting more subjects of cognitive impairment to test and revise the existing methods, and considering adequately the dynamic relationships among different brain regions.

Author Contributions

DW: designed the study and wrote this paper; ZW and YZ: analyzed literatures and wrote this paper; GL and XZ: revised this paper; WH: designed the study.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This research was funded by National Natural Science Foundation of China (61503326), Natural Science Foundation of Hebei Province in China (F2016203343), China Postdoctoral Science Foundation (2015M581317) and Doctorial Foundation of Yanshan University in China (B900).

References