Top

Published in:

Open Access 01-07-2022 | Electroencephalography | Original Article

Multimodal Autoencoder Predicts fNIRS Resting State From EEG Signals

Authors: Parikshat Sirpal, Rafat Damseh, Ke Peng, Dang Khoa Nguyen, Frédéric Lesage

Published in: Neuroinformatics | Issue 3/2022

Abstract

In this work, we introduce a deep learning architecture for evaluation on multimodal electroencephalographic (EEG) and functional near-infrared spectroscopy (fNIRS) recordings from 40 epileptic patients. Long short-term memory units and convolutional neural networks are integrated within a multimodal sequence-to-sequence autoencoder. The trained neural network predicts fNIRS signals from EEG, sans a priori, by hierarchically extracting deep features from EEG full spectra and specific EEG frequency bands. Results show that higher frequency EEG ranges are predictive of fNIRS signals with the gamma band inputs dominating fNIRS prediction as compared to other frequency envelopes. Seed based functional connectivity validates similar patterns between experimental fNIRS and our model’s fNIRS reconstructions. This is the first study that shows it is possible to predict brain hemodynamics (fNIRS) from encoded neural data (EEG) in the resting human epileptic brain based on power spectrum amplitude modulation of frequency oscillations in the context of specific hypotheses about how EEG frequency bands decode fNIRS signals.

Aalkjær, C., Boedtkjer, D., & Matchkov, V. (2011). Vasomotion – what is currently thought? In Acta Physiologica. https://doi.org/10.1111/j.1748-1716.2011.02320.x

Aasted, C. M., Yücel, M. A., Cooper, R. J., Dubb, J., Tsuzuki, D., Becerra, L., Petkov, M. P., Borsook, D., Dan, I., & Boas, D. A. (2015). Anatomical guidance for functional near-infrared spectroscopy: AtlasViewer tutorial. Neurophotonics. https://doi.org/10.1117/1.nph.2.2.020801CrossRefPubMedPubMedCentral

Amzica, F., & Steriade, M. (1998). Electrophysiological correlates of sleep delta waves. Electroencephalography and Clinical Neurophysiology, 107(2), 69–83.CrossRef

Barlow, J. S., Kamp, A., Morton, H. B., Ripoche, A., Shipton, H., & Tchavdarov, D. B. (1974). EEG instrumentation standards: Report of the Committee on EEG Instrumentation Standards of the International Federation of Societies for Electroencephalography and Clinical Neurophysiology. Electroencephalography and Clinical Neurophysiology, 37(5), 549–553. https://doi.org/10.1016/0013-4694(74)90100-XCrossRef

Baytas, I. M., Xiao, C., Zhang, X., Wang, F., Jain, A. K., & Zhou, J. (2017). Patient subtyping via time-aware LSTM networks. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. https://doi.org/10.1145/3097983.3097997CrossRef

Bettus, G., Guedj, E., Joyeux, F., Confort-Gouny, S., Soulier, E., Laguitton, V., Cozzone, P. J., Chauvel, P., Ranjeva, J. P., Bartolomei, F., & Guye, M. (2009). Decreased basal fMRI functional connectivity in epileptogenic networks and contralateral compensatory mechanisms. Human Brain Mapping, 30(5), 1580–1591. https://doi.org/10.1002/hbm.20625CrossRefPubMed

Canolty, R. T., Edwards, E., Dalal, S. S., Soltani, M., Nagarajan, S. S., Kirsch, H. E., Berger, M. S., Barbare, N. M., & Knight, R. T. (2006). High gamma power is phase-locked to theta oscillations in human neocortex. Science. https://doi.org/10.1126/science.1128115CrossRefPubMedPubMedCentral

Chiarelli, A. M., Zappasodi, F., Di Pompeo, F., & Merla, A. (2017). Simultaneous functional near-infrared spectroscopy and electroencephalography for monitoring of human brain activity and oxygenation: A review. Neurophotonics. https://doi.org/10.1117/1.nph.4.4.041411CrossRefPubMedPubMedCentral

Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. https://doi.org/10.3115/v1/d14-1179

Chung, Y. A., Wu, C. C., Shen, C. H., Lee, H. Y., & Lee, L. S. (2016). Audio Word2Vec: Unsupervised learning of audio segment representations using sequence-to-sequence autoencoder. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. https://doi.org/10.21437/Interspeech.2016-82

Citerio, G., Oddo, M., & Taccone, F. S. (2015). Recommendations for the use of multimodal monitoring in the neurointensive care unit. In Current Opinion in Critical Care (Vol. 21, Issue 2, pp. 113–119). Lippincott Williams and Wilkins. https://doi.org/10.1097/MCC.0000000000000179

Connolly, M., Vespa, P., Pouratian, N., Gonzalez, N. R., & Hu, X. (2015). Characterization of the relationship between intracranial pressure and electroencephalographic monitoring in burst-suppressed patients. Neurocritical Care, 22(2), 212–220. https://doi.org/10.1007/s12028-014-0059-8CrossRefPubMedPubMedCentral

Curtin, A., Tong, S., Sun, J., Wang, J., Onaral, B., & Ayaz, H. (2019). A systematic review of integrated functional near-infrared spectroscopy (fNIRS) and transcranial magnetic stimulation (TMS) studies. Frontiers in Neuroscience. https://doi.org/10.3389/fnins.2019.00084CrossRefPubMedPubMedCentral

Czisch, M., Wehrle, R., Kaufmann, C., Wetter, T. C., Holsboer, F., Pollmächer, T., & Auer, D. P. (2004). Functional MRI during sleep: BOLD signal decreases and their electrophysiological correlates. European Journal of Neuroscience. https://doi.org/10.1111/j.1460-9568.2004.03518.xCrossRefPubMed

Dash, D., Dash, C., Primrose, S., Hernandez-Ronquillo, L., Moien-Afshari, F., Ladino, L. D., Appendino, J. P., Mazepa, L., Elliott, C., Mirsattari, S. M., Federico, P., Bui, E., Hunter, G., Ramachandran Nair, R., Sharma, R., Melendres, P., Nikkel, J., Nguyen, D. K., Almubarak, S., … Téllez-Zenteno, J. F. (2017). Update on minimal standards for electroencephalography in Canada: A review by the Canadian Society of Clinical Neurophysiologists. In Canadian Journal of Neurological Sciences (Vol. 44, Issue 6, pp. 631–642). Cambridge University Press. https://doi.org/10.1017/cjn.2017.217

De Luca, M., Beckmann, C. F., De Stefano, N., Matthews, P. M., & Smith, S. M. (2006). fMRI resting state networks define distinct modes of long-distance interactions in the human brain. NeuroImage, 29(4), 1359–1367. https://doi.org/10.1016/j.neuroimage.2005.08.035CrossRefPubMed

DeBettencourt, M. T., Cohen, J. D., Lee, R. F., Norman, K. A., & Turk-Browne, N. B. (2015). Closed-loop training of attention with real-time brain imaging. Nature Neuroscience. https://doi.org/10.1038/nn.3940CrossRefPubMedPubMedCentral

Deco, G., Jirsa, V. K., & McIntosh, A. R. (2011). Emerging concepts for the dynamical organization of resting-state activity in the brain. In Nature Reviews Neuroscience (Vol. 12, Issue 1, pp. 43–56). Nature Publishing Group. https://doi.org/10.1038/nrn2961

Dhamija, R., Moseley, B. D., Cascino, G. D., & Wirrell, E. C. (2011). A population-based study of long-term outcome of epilepsy in childhood with a focal or hemispheric lesion on neuroimaging. Epilepsia, 52(8), 1522–1526. https://doi.org/10.1111/j.1528-1167.2011.03192.xCrossRefPubMedPubMedCentral

Dutta, A., Jacob, A., Chowdhury, S. R., Das, A., & Nitsche, M. A. (2015). EEG-NIRS based assessment of neurovascular coupling during anodal transcranial direct current stimulation - a stroke case series. Journal of Medical Systems, 39(4), 1–9. https://doi.org/10.1007/s10916-015-0205-7CrossRef

Ebisch, B., Schmidt, K. E., Niessing, M., Singer, W., Galuske, R. A. W., & Niessing, J. (2005). Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science.

Fair, D. A., Nigg, J. T., Iyer, S., Bathula, D., Mills, K. L., Dosenbach, N. U. F., Schlaggar, B. L., Mennes, M., Gutman, D., Bangaru, S., Buitelaar, J. K., Dickstein, D. P., Martino, A. D., Kennedy, D. N., Kelly, C., Luna, B., Schweitzer, J. B., Velanova, K., Wang, Y. F., & Milham, M. P. (2013). Distinct neural signatures detected for ADHD subtypes after controlling for micro-movements in resting state functional connectivity MRI data. Frontiers in Systems Neuroscience. https://doi.org/10.3389/fnsys.2012.00080CrossRefPubMedPubMedCentral

Fazli, S., Mehnert, J., Steinbrink, J., Curio, G., Villringer, A., Müller, K. R., & Blankertz, B. (2012). Enhanced performance by a hybrid NIRS-EEG brain computer interface. NeuroImage, 59(1), 519–529. https://doi.org/10.1016/j.neuroimage.2011.07.084CrossRefPubMed

Fishburn, F. A., Norr, M. E., Medvedev, A. V., & Vaidya, C. J. (2014). Sensitivity of fNIRS to cognitive state and load. Frontiers in Human Neuroscience. https://doi.org/10.3389/fnhum.2014.00076CrossRefPubMedPubMedCentral

Foreman, B., & Claassen, J. (2012). Quantitative EEG for the detection of brain ischemia. Critical Care. https://doi.org/10.1186/cc11230CrossRefPubMedPubMedCentral

Franaszczuk, P. J., Kudela, P., & Bergey, G. K. (2003). External excitatory stimuli can terminate bursting in neural network models. Epilepsy Research, 53(1–2), 65–80. https://doi.org/10.1016/S0920-1211(02)00248-6CrossRefPubMed

Freeman, W. J., Burke, B. C., & Holmes, M. D. (2003). Aperiodic phase re-setting in scalp EEG of beta-gamma oscillations by state transitions at alpha-theta rates. Human Brain Mapping. https://doi.org/10.1002/hbm.10120CrossRefPubMedPubMedCentral

Gagnon, L., Cooper, R. J., Yücel, M. A., Perdue, K. L., Greve, D. N., & Boas, D. A. (2012). Short separation channel location impacts the performance of short channel regression in NIRS. NeuroImage. https://doi.org/10.1016/j.neuroimage.2011.08.095CrossRefPubMed

Geng, S., Liu, X., Biswal, B. B., & Niu, H. (2017). Effect of resting-state fNIRS scanning duration on functional brain connectivity and graph theory metrics of brain network. Frontiers in Neuroscience, 11(JUL). https://doi.org/10.3389/fnins.2017.00392

Girouard, H., & Iadecola, C. (2006). Neurovascular coupling in the normal brain and in hypertension, stroke, and Alzheimer disease. Journal of Applied Physiology, 100(1), 328–335.CrossRef

Goldman, R. I., Stern, J. M., Engel, J., Jr., & Cohen, M. S. (2002). Simultaneous EEG and fMRI of the alpha rhythm. NeuroReport, 13(18), 2487.CrossRef

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press.

Gramfort, A., Luessi, M., Larson, E., Engemann, D. A., Strohmeier, D., Brodbeck, C., Parkkonen, L., & Hämäläinen, M. S. (2014). MNE software for processing MEG and EEG data. NeuroImage. https://doi.org/10.1016/j.neuroimage.2013.10.027CrossRefPubMed

Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017). LSTM: A Search Space Odyssey. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2016.2582924CrossRefPubMed

Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., & Wierstra, D. (2015). DRAW: A recurrent neural network for image generation. 32nd International Conference on Machine Learning, ICML 2015.

Gross, D. W., & Gotman, J. (1999). Correlation of high-frequency oscillations with the sleep-wake cycle and cognitive activity in humans. Neuroscience. https://doi.org/10.1016/S0306-4522(99)00343-7CrossRefPubMed

He, B., & Liu, Z. (2008). Multimodal functional neuroimaging: integrating functional MRI and EEG/MEG. IEEE Reviews in Biomedical Engineering, 1(2008), 23–40. https://doi.org/10.1109/RBME.2008.2008233CrossRefPubMedPubMedCentral

Helbok, R., & Claassen, J. (2013). Multimodal invasive monitoring in status epilepticus: What is the evidence it has a place? Epilepsia, 54(SUPPL. 6), 57–60. https://doi.org/10.1111/epi.12279CrossRefPubMed

Hofle, N., Paus, T., Reutens, D., Fiset, P., Gotman, J., Evans, A. C., & Jones, B. E. (1997). Regional cerebral blood flow changes as a function of delta and spindle activity during slow wave sleep in humans. Journal of Neuroscience. https://doi.org/10.1523/jneurosci.17-12-04800.1997CrossRefPubMed

Honda, Y., Nakamura, S., Ogawa, K., Yoshino, R., Tobler, P. N., Nishimura, Y., & Tsutsui, K. I. (2021). Changes in beta and high-gamma power in resting-state electrocorticogram induced by repetitive transcranial magnetic stimulation of primary motor cortex in unanesthetized macaque monkeys. Neuroscience Research. https://doi.org/10.1016/j.neures.2021.02.002CrossRefPubMed

Hong, K. S., Naseer, N., & Kim, Y. H. (2015). Classification of prefrontal and motor cortex signals for three-class fNIRS-BCI. Neuroscience Letters, 587, 87–92. https://doi.org/10.1016/j.neulet.2014.12.029CrossRefPubMed

Hughes, J. R. (2008). Gamma, fast, and ultrafast waves of the brain: Their relationships with epilepsy and behavior. In Epilepsy and Behavior (Vol. 13, Issue 1, pp. 25–31). Academic Press. https://doi.org/10.1016/j.yebeh.2008.01.011

Huppert, T. J., Diamond, S. G., Franceschini, M. A., & Boas, D. A. (2009). HomER: A review of time-series analysis methods for near-infrared spectroscopy of the brain. Applied Optics, 48(10), D280–D298.CrossRef

Jia, X., & Kohn, A. (2011). Gamma rhythms in the brain. PLoS Biology, 9(4), 2–5. https://doi.org/10.1371/journal.pbio.1001045CrossRef

Jobsis, F. F. (1977). Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. Science, 198(4323), 1264–1267.CrossRef

Julien, C. (2006). The enigma of Mayer waves: Facts and models. Cardiovascular Research. https://doi.org/10.1016/j.cardiores.2005.11.008CrossRefPubMed

Khan, M. J., & Hong, K. -S. (2015). Passive BCI based on drowsiness detection: An fNIRS study. Biomedical Optics Express, 6(10), 4063. https://doi.org/10.1364/boe.6.004063CrossRefPubMedPubMedCentral

Khan, M. J., & Hong, K. S. (2017). Hybrid EEG-FNIRS-based eight-command decoding for BCI: Application to quadcopter control. Frontiers in Neurorobotics, 11, 6. https://doi.org/10.3389/fnbot.2017.00006CrossRefPubMedPubMedCentral

Knyazev, G. G. (2012). EEG delta oscillations as a correlate of basic homeostatic and motivational processes. Neuroscience & Biobehavioral Reviews, 36(1), 677–695.CrossRef

Kobayashi, E., Bagshaw, A. P., Grova, C., Dubeau, F., & Gotman, J. (2006). Negative BOLD responses to epileptic spikes. Human Brain Mapping, 27(6), 488–497. https://doi.org/10.1002/hbm.20193CrossRefPubMed

Koch, S. P., Koendgen, S., Bourayou, R., Steinbrink, J., & Obrig, H. (2008). Individual alpha-frequency correlates with amplitude of visual evoked potential and hemodynamic response. NeuroImage, 41(2), 233–242.CrossRef

Kocsis, L., Herman, P., & Eke, A. (2006). The modified Beer-Lambert law revisited. Physics in Medicine and Biology, 51(5). https://doi.org/10.1088/0031-9155/51/5/N02

Kohno, S., Miyai, I., Seiyama, A., Oda, I., Ishikawa, A., Tsuneishi, S., Amita, T., & Shimizu, K. (2007). Removal of the skin blood flow artifact in functional near-infrared spectroscopic imaging data through independent component analysis. Journal of Biomedical Optics. https://doi.org/10.1117/12814249CrossRefPubMed

Kotliar, K., Hauser, C., Ortner, M., Muggenthaler, C., Diehl-Schmid, J., Angermann, S., Hapfelmeier, A., Schmaderer, C., & Grimmer, T. (2017). Altered neurovascular coupling as measured by optical imaging: A biomarker for Alzheimer’s disease. Scientific Reports, 7(1), 1–11. https://doi.org/10.1038/s41598-017-13349-5CrossRef

Laufs, H., Krakow, K., Sterzer, P., Eger, E., Beyerle, A., Salek-Haddadi, A., & Kleinschmidt, A. (2003). Electroencephalographic signatures of attentional and cognitive default modes in spontaneous brain activity fluctuations at rest. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.1831638100CrossRefPubMedPubMedCentral

Le Roux, P. (2013). Physiological monitoring of the severe traumatic brain injury patient in the intensive care unit. Current Neurology and Neuroscience Reports, 13(3), 331.CrossRef

Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature. https://doi.org/10.1038/nature14539CrossRefPubMed

Lemieux, L., Krakow, K., & Fish, D. R. (2001). Comparison of spike-triggered functional MRI BOLD activation and EEG dipole model localization. NeuroImage. https://doi.org/10.1006/nimg.2001.0896CrossRefPubMed

Li, Z., Liu, H., Liao, X., Xu, J., Liu, W., Tian, F., He, Y., & Niu, H. (2015). Dynamic functional connectivity revealed by resting-state functional near-infrared spectroscopy. Biomedical Optics Express. https://doi.org/10.1364/boe.6.002337CrossRefPubMedPubMedCentral

Lindauer, U., Dirnagl, U., Füchtemeier, M., Böttiger, C., Offenhauser, N., Leithner, C., & Royl, G. (2010). Pathophysiological interference with neurovascular coupling–when imaging based on hemoglobin might go blind. Frontiers in Neuroenergetics, 2, 25.CrossRef

Lipton, Z. C., Kale, D. C., Elkan, C., & Wetzel, R. (2016). Learning to diagnose with LSTM recurrent neural networks. 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings.

Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature. https://doi.org/10.1038/35084005CrossRefPubMed

Lövblad, K. O., Thomas, R., Jakob, P. M., Scammell, T., Bassetti, C., Griswold, M., Ives, J., Matheson, J., Edelman, R. R., & Warach, S. (1999). Silent functional magnetic resonance imaging demonstrates focal activation in rapid eye movement sleep. Neurology. https://doi.org/10.1212/wnl.53.9.2193CrossRefPubMed

Lu, C. M., Zhang, Y. J., Biswal, B. B., Zang, Y. F., Peng, D. L., & Zhu, C. Z. (2010). Use of fNIRS to assess resting state functional connectivity. Journal of Neuroscience Methods, 186(2), 242–249. https://doi.org/10.1016/j.jneumeth.2009.11.010CrossRefPubMed

Luo, C., Li, Q., Lai, Y., Xia, Y., Qin, Y., Liao, W., Li, S., Zhou, D., Yao, D., & Gong, Q. (2011). Altered functional connectivity in default mode network in absence epilepsy: A resting-state fMRI study. Human Brain Mapping, 32(3), 438–449. https://doi.org/10.1002/hbm.21034CrossRefPubMed

Luong, M. -T., Le, Q. V., Sutskever, I., Vinyals, O., & Kaiser, L. (2015). Multi-task Sequence to Sequence Learning. c, 1–10. http://arxiv.org/abs/1511.06114

Maas, A. I. R., Menon, D. K., Lingsma, H. F., Pineda, J. A., Sandel, M. E., & Manley, G. T. (2012). Re-orientation of clinical research in traumatic brain injury: Report of an international workshop on comparative effectiveness research. Journal of Neurotrauma, 29(1), 32–46. https://doi.org/10.1089/neu.2010.1599CrossRefPubMedPubMedCentral

Mantini, D., Perrucci, M. G., Del Gratta, C., Romani, G. L., & Corbetta, M. (2007). Electrophysiological signatures of resting state networks in the human brain. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.0700668104CrossRefPubMedPubMedCentral

Martinez-Montes, E., Valdes-Sosa, P. A., Miwakeichi, F., Goldman, R. I., & Cohen, M. S. (2004). Concurrent EEG/fMRI analysis by multiway partial least squares. NeuroImage, 22(3), 1023–1034. https://doi.org/10.1016/j.neuroimage.2004.03.038CrossRefPubMed

McKenna, T. M., McMullen, T. A., & Shlesinger, M. F. (1994). The brain as a dynamic physical system. Neuroscience, 60(3), 587–605. https://doi.org/10.1016/0306-4522(94)90489-8CrossRefPubMed

Merity, S., Keskar, N. S., & Socher, R. (2018). Regularizing and optimizing LSTM language models. 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings.

Merker, B. H. (2016). Cortical gamma oscillations: Details of their genesis preclude a role in cognition. Frontiers in Computational Neuroscience. https://doi.org/10.3389/fncom.2016.00078CrossRefPubMedPubMedCentral

Mikolov, T., Karafiát, M., Burget, L., Jan, C., & Khudanpur, S. (2010). Recurrent neural network based language model. Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010.

Miller, C. M. (2012). Update on multimodality monitoring. Current Neurology and Neuroscience Reports. https://doi.org/10.1007/s11910-012-0274-7CrossRefPubMed

Moshé, S. L., Perucca, E., Ryvlin, P., & Tomson, T. (2015). Epilepsy: New advances. In The Lancet (Vol. 385, Issue 9971, pp. 884–898). Lancet Publishing Group. https://doi.org/10.1016/S0140-6736(14)60456-6

Mukamel, R., Gelbard, H., Arieli, A., Hasson, U., Fried, I., & Malach, R. (2005). Neuroscience: Coupling between neuronal firing, field potentials, and fMRI in human auditory cortex. Science. https://doi.org/10.1126/science.1110913CrossRefPubMed

Müller-Putz, G. R. (2020). Electroencephalography. Handbook of Clinical Neurology. https://doi.org/10.1016/B978-0-444-63934-9.00018-4CrossRefPubMed

Murta, T., Leite, M., Carmichael, D. W., Figueiredo, P., & Lemieux, L. (2015). Electrophysiological correlates of the BOLD signal for EEG-informed fMRI. Human Brain Mapping, 36(1), 391–414. https://doi.org/10.1002/hbm.22623CrossRefPubMed

Nicholson, A. A., Ros, T., Frewen, P. A., Densmore, M., Théberge, J., Kluetsch, R. C., Jetly, R., & Lanius, R. A. (2016). Alpha oscillation neurofeedback modulates amygdala complex connectivity and arousal in posttraumatic stress disorder. NeuroImage: Clinical. https://doi.org/10.1016/j.nicl.2016.07.006

Niessing, J., Ebisch, B., Schmidt, K. E., Niessing, M., Singer, W., & Galuske, R. A. W. (2005). Neuroscience: Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science, 309(5736), 948–951. https://doi.org/10.1126/science.1110948CrossRefPubMed

Nikulin, V. V., Fedele, T., Mehnert, J., Lipp, A., Noack, C., Steinbrink, J., & Curio, G. (2014). Monochromatic Ultra-Slow (~ 0.1 Hz) Oscillations in the human electroencephalogram and their relation to hemodynamics. NeuroImage, 97, 71–80. https://doi.org/10.1016/J.NEUROIMAGE.2014.04.008CrossRefPubMed

Niu, H., & He, Y. (2014). Resting-state functional brain connectivity: Lessons from functional near-infrared spectroscopy. The Neuroscientist. https://doi.org/10.1177/1073858413502707CrossRefPubMed

Ogawa, S., Tank, D. W., Menon, R., Ellermann, J. M., Kim, S. G., Merkle, H., & Ugurbil, K. (1992). Intrinsic signal changes accompanying sensory stimulation: Functional brain mapping with magnetic resonance imaging. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.89.13.5951CrossRefPubMedPubMedCentral

Palva, J. M., Monto, S., Kulashekhar, S., & Palva, S. (2010). Neuronal synchrony reveals working memory networks and predicts individual memory capacity. Proceedings of the National Academy of Sciences of the United States of America, 107(16), 7580–7585. https://doi.org/10.1073/pnas.0913113107CrossRefPubMedPubMedCentral

Park, Y., Luo, L., Parhi, K. K., & Netoff, T. (2011). Seizure prediction with spectral power of EEG using cost-sensitive support vector machines. Epilepsia, 52(10), 1761–1770. https://doi.org/10.1111/j.1528-1167.2011.03138.xCrossRefPubMed

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research.

Peng, K., Nguyen, D. K., Tayah, T., Vannasing, P., Tremblay, J., Sawan, M., Lassonde, M., Lesage, F., & Pouliot, P. (2014). FNIRS-EEG study of focal interictal epileptiform discharges. Epilepsy Research. https://doi.org/10.1016/j.eplepsyres.2013.12.011CrossRefPubMed

Phillips, A. A., Squair, J. W., Sayenko, D. G., Edgerton, V. R., Gerasimenko, Y., & Krassioukov, A. V. (2018). An autonomic neuroprosthesis: Noninvasive electrical spinal cord stimulation restores autonomic cardiovascular function in individuals with spinal cord injury. Journal of Neurotrauma, 35(3), 446–451. https://doi.org/10.1089/neu.2017.5082CrossRefPubMedPubMedCentral

Pizzo, F., Frauscher, B., Ferrari-Marinho, T., Amiri, M., Dubeau, F., & Gotman, J. (2016). Detectability of fast ripples (>250 Hz) on the scalp EEG: A proof-of-principle study with subdermal electrodes. Brain Topography. https://doi.org/10.1007/s10548-016-0481-7CrossRefPubMed

Pressl, C., Brandner, P., Schaffelhofer, S., Blackmon, K., Dugan, P., Holmes, M., Thesen, T., Kuzniecky, R., Devinsky, O., & Freiwald, W. A. (2019). Resting state functional connectivity patterns associated with pharmacological treatment resistance in temporal lobe epilepsy. Epilepsy Research. https://doi.org/10.1016/j.eplepsyres.2018.11.002CrossRefPubMed

Quaresima, V., & Ferrari, M. (2019). A mini-review on functional near-infrared spectroscopy (fNIRS): Where do we stand, and where should we go? Photonics, 6(3), 87. https://doi.org/10.3390/photonics6030087CrossRef

Ray, S., & Maunsell, J. H. R. (2010). Differences in gamma frequencies across visual cortex restrict their possible use in computation. Neuron, 67(5), 885–896. https://doi.org/10.1016/j.neuron.2010.08.004CrossRefPubMedPubMedCentral

Richardson, M. P. (2012). Large scale brain models of epilepsy: Dynamics meets connectomics. Journal of Neurology, Neurosurgery and Psychiatry, 83(12), 1238–1248. https://doi.org/10.1136/jnnp-2011-301944CrossRefPubMed

Rojas, G. M., Alvarez, C., Montoya, C. E., de la Iglesia-Vayá, M., Cisternas, J. E., & Gálvez, M. (2018). Study of resting-state functional connectivity networks using EEG electrodes position as seed. Frontiers in Neuroscience. https://doi.org/10.3389/fnins.2018.00235CrossRefPubMedPubMedCentral

Ros, T., Baars, B. J., Lanius, R. A., & Vuilleumier, P. (2014). Tuning pathological brain oscillations with neurofeedback: A systems neuroscience framework. Frontiers in Human Neuroscience, 8, 1008. https://doi.org/10.3389/fnhum.2014.01008CrossRefPubMedPubMedCentral

Salek-Haddadi, A., Friston, K. J., Lemieux, L., & Fish, D. R. (2003). Studying spontaneous EEG activity with fMRI. Brain Research Reviews. https://doi.org/10.1016/S0165-0173(03)00193-0CrossRefPubMed

Sassaroli, A., Pierro, M., Bergethon, P. R., & Fantini, S. (2012). Low-frequency spontaneous oscillations of cerebral hemodynamics investigated with near-infrared spectroscopy: A review. IEEE Journal on Selected Topics in Quantum Electronics. https://doi.org/10.1109/JSTQE.2012.2183581CrossRef

Schmidt, J. M., & De Georgia, M. (2014). Multimodality Monitoring: Informatics, Integration Data Display and Analysis The Participants in the International Multidisciplinary Consensus Conference on Multimodality Monitoring. Neurocritical Care. https://doi.org/10.1007/s12028-014-0037-1

Scholkmann, F., Kleiser, S., Metz, A. J., Zimmermann, R., Pavia, J. M., Wolf, U., & Wolf, M. (2014). A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology. NeuroImage, 85, 6–27.CrossRef

Schwab, K., Skupin, H., Eiselt, M., Walther, M., Voss, A., & Witte, H. (2009). Coordination of the EEG and the heart rate of preterm neonates during quiet sleep. Neuroscience Letters. https://doi.org/10.1016/j.neulet.2009.09.031CrossRefPubMed

Sharbrough, F. W., Messick, J. M., & Sundt, T. M. (1973). Correlation of continuous electroencephalograms with cerebral blood flow measurements during carotid endarterectomy. Stroke. https://doi.org/10.1161/01.STR.4.4.674CrossRefPubMed

Shen, H. H. (2015). Core concept: Resting-state connectivity. Proceedings of the National Academy of Sciences of the United States of America. https://doi.org/10.1073/pnas.1518785112CrossRefPubMedPubMedCentral

Shmuel, A., & Leopold, D. A. (2008). Neuronal correlates of spontaneous fluctuations in fMRI signals in monkey visual cortex: Implications for functional connectivity at rest. Human Brain Mapping, 29(7), 751–761. https://doi.org/10.1002/hbm.20580CrossRefPubMedPubMedCentral

Siegel, J. S., Snyder, A. Z., Ramsey, L., Shulman, G. L., & Corbetta, M. (2016). The effects of hemodynamic lag on functional connectivity and behavior after stroke. Journal of Cerebral Blood Flow and Metabolism, 36(12), 2162–2176. https://doi.org/10.1177/0271678X15614846CrossRefPubMed

Sigala, R., Haufe, S., Roy, D., Dinse, H. R., & Ritter, P. (2014). The role of alpha-rhythm states in perceptual learning: Insights from experiments and computational models. Frontiers in Computational Neuroscience, 8, 1–19. https://doi.org/10.3389/fncom.2014.00036CrossRef

Singh, K. D. (2012). Which “neural activity” do you mean? fMRI, MEG, oscillations and neurotransmitters. NeuroImage. https://doi.org/10.1016/j.neuroimage.2012.01.028CrossRefPubMed

Sirpal, P., Kassab, A., Pouliot, P., & Nguyen, D. K. (2019). fNIRS improves seizure detection in multimodal EEG-fNIRS recordings. Journal of Biomedical Optics, 24(05), 1. https://doi.org/10.1117/1.jbo.24.5.051408CrossRefPubMed

Sitaram, R., Ros, T., Stoeckel, L., Haller, S., Scharnowski, F., Lewis-Peacock, J., Weiskopf, N., Blefari, M. L., Rana, M., Oblak, E., Birbaumer, N., & Sulzer, J. (2017). Closed-loop brain training: The science of neurofeedback. Nature Reviews Neuroscience. https://doi.org/10.1038/nrn.2016.164CrossRefPubMed

Sitnikova, E., Hramov, A. E., Grubov, V., & Koronovsky, A. A. (2016). Rhythmic activity in EEG and sleep in rats with absence epilepsy. Brain Research Bulletin. https://doi.org/10.1016/j.brainresbull.2015.11.012CrossRefPubMed

Socher, R., Pennington, J., Huang, E. H., Ng, A. Y., & Manning, C. D. (2011). Semi-supervised recursive autoencoders for predicting sentiment distributions. EMNLP 2011 - Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference.

Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015). Unsupervised learning of video representations using LSTMs. 32nd International Conference on Machine Learning, ICML 2015.

Strangman, G., Boas, D. A., & Sutton, J. P. (2002). Non-invasive neuroimaging using near-infrared light. Biological Psychiatry. https://doi.org/10.1016/S0006-3223(02)01550-0CrossRefPubMed

Sumiyoshi, A., Suzuki, H., Ogawa, T., Riera, J. J., Shimokawa, H., & Kawashima, R. (2012). Coupling between gamma oscillation and fMRI signal in the rat somatosensory cortex: Its dependence on systemic physiological parameters. NeuroImage. https://doi.org/10.1016/j.neuroimage.2011.12.082CrossRefPubMed

Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems, 4(January), 3104–3112.

Thair, H., Holloway, A. L., Newport, R., & Smith, A. D. (2017). Transcranial direct current stimulation (tDCS): A Beginner’s guide for design and implementation. Frontiers in Neuroscience, 11, 641. https://doi.org/10.3389/fnins.2017.00641CrossRefPubMedPubMedCentral

Tong, Y., Hocke, L. M., Licata, S. C., & Frederick, B. D. (2012). Low-frequency oscillations measured in the periphery with near-infrared spectroscopy are strongly correlated with blood oxygen level-dependent functional magnetic resonance imaging signals. Journal of Biomedical Optics, 17(10), 1060041. https://doi.org/10.1117/1.jbo.17.10.106004CrossRef

Tracy, J. I., & Doucet, G. E. (2015). Resting-state functional connectivity in epilepsy: Growing relevance for clinical decision making. Current Opinion in Neurology. https://doi.org/10.1097/WCO.0000000000000178CrossRefPubMed

Truong, N. D., Kuhlmann, L., Bonyadi, M. R., & Kavehei, O. (2018). Semi-supervised Seizure Prediction with Generative Adversarial Networks. 1–6. http://arxiv.org/abs/1806.08235

Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P. A. (2008). Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th International Conference on Machine Learning. https://doi.org/10.1145/1390156.1390294

Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., & Manzagol, P. A. (2010). Stacked denoising autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. Journal of Machine Learning Research.

Wang, J., Dong, Q., & Niu, H. (2017). The minimum resting-state fNIRS imaging duration for accurate and stable mapping of brain connectivity network in children. Scientific Reports, 7(1). https://doi.org/10.1038/s41598-017-06340-7

Wang, Y., Huang, M., Zhao, L., & Zhu, X. (2016). Attention-based LSTM for aspect-level sentiment classification. EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings. https://doi.org/10.18653/v1/d16-1058

Wang, Z., Lu, G., Zhang, Z., Zhong, Y., Jiao, Q., Zhang, Z., Tan, Q., Tian, L., Chen, G., Liao, W., Li, K., & Liu, Y. (2011). Altered resting state networks in epileptic patients with generalized tonic-clonic seizures. Brain Research, 1374, 134–141. https://doi.org/10.1016/j.brainres.2010.12.034CrossRefPubMed

Welch, P. D. (1967). The Use of Fast Fourier Transform for the Estimation of Power Spectra: A Method Based on Time Averaging Over Short, Modified Periodograms. IEEE Transactions on Audio and Electroacoustics. https://doi.org/10.1109/TAU.1967.1161901

Whittingstall, K., & Logothetis, N. K. (2009). Frequency-band coupling in surface EEG reflects spiking activity in monkey visual cortex. Neuron. https://doi.org/10.1016/j.neuron.2009.08.016CrossRefPubMed

Woermann, F. G., & Vollmar, C. (2009). Clinical MRI in children and adults with focal epilepsy: A critical review. Epilepsy and Behavior, 15(1), 40–49. https://doi.org/10.1016/j.yebeh.2009.02.032CrossRefPubMed

Yücel, M. A., Selb, J., Aasted, C. M., Lin, P.-Y., Borsook, D., Becerra, L., & Boas, D. A. (2016). Mayer waves reduce the accuracy of estimated hemodynamic response functions in functional near-infrared spectroscopy. Biomedical Optics Express, 7(8), 3078. https://doi.org/10.1364/boe.7.003078CrossRefPubMedPubMedCentral

Zhang, H., Zhang, Y. J., Lu, C. M., Ma, S. Y., Zang, Y. F., & Zhu, C. Z. (2010a). Functional connectivity as revealed by independent component analysis of resting-state fNIRS measurements. NeuroImage. https://doi.org/10.1016/j.neuroimage.2010.02.080CrossRefPubMed

Zhang, W. (2018). Generative Adversarial Nets for Information Retrieval. The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval - SIGIR ’18, 1375–1378. https://doi.org/10.1145/3209978.3210184

Zhang, X., Pan, W. J., & Keilholz, S. D. (2020). The relationship between BOLD and neural activity arises from temporally sparse events. NeuroImage, 207, 116390. https://doi.org/10.1016/j.neuroimage.2019.116390CrossRefPubMed

Zhang, Z., Lu, G., Zhong, Y., Tan, Q., Liao, W., Wang, Z., Wang, Z., Li, K., Chen, H., & Liu, Y. (2010b). Altered spontaneous neuronal activity of the default-mode network in mesial temporal lobe epilepsy. Brain Research, 1323, 152–160. https://doi.org/10.1016/j.brainres.2010.01.042CrossRefPubMed

Zhang, Z., Lu, G., Zhong, Y., Tan, Q., Yang, Z., Liao, W., Chen, Z., Shi, J., & Liu, Y. (2009). Impaired attention network in temporal lobe epilepsy: A resting FMRI study. Neuroscience Letters, 458(3), 97–101. https://doi.org/10.1016/j.neulet.2009.04.040CrossRefPubMed

Zhao, F., Feng, J., Zhao, J., Yang, W., & Yan, S. (2018). Robust LSTM-Autoencoders for Face De-Occlusion in the Wild. IEEE Transactions on Image Processing. https://doi.org/10.1109/TIP.2017.2771408CrossRefPubMed

Title: Multimodal Autoencoder Predicts fNIRS Resting State From EEG Signals
Authors: Parikshat Sirpal
Rafat Damseh
Ke Peng
Dang Khoa Nguyen
Frédéric Lesage
Publication date: 01-07-2022
Publisher: Springer US
Keywords: Electroencephalography
Epilepsy
Published in: Neuroinformatics / Issue 3/2022
Print ISSN: 1539-2791
Electronic ISSN: 1559-0089
DOI: https://doi.org/10.1007/s12021-021-09538-3

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Multimodal Autoencoder Predicts fNIRS Resting State From EEG Signals

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 3/2022

Building Models of Functional Interactions Among Brain Domains that Encode Varying Information Complexity: A Schizophrenia Case Study

A Practical Workflow for Organizing Clinical Intraoperative and Long-term iEEG Data in BIDS

Correction to: High‑throughput Analysis of Synaptic Activity in Electrically Stimulated Neuronal Cultures

The Neuron Phenotype Ontology: A FAIR Approach to Proposing and Classifying Neuronal Types

Extreme Learning Machine Design for Dealing with Unrepresentative Features

Intelligible Models for HealthCare: Predicting the Probability of 6-Month Unfavorable Outcome in Patients with Ischemic Stroke