Top

BMC Neurology

Published in:

Open Access 01-12-2023 | Mild Neurocognitive Disorder | Research

Mental workload classification using convolutional neural networks based on fNIRS-derived prefrontal activity

Author: Jin-Hyuck Park

Published in: BMC Neurology | Issue 1/2023

Abstract

Background

Functional near-infrared spectroscopy (fNIRS) is a tool to assess brain activity during cognitive testing. Despite its usefulness, its feasibility in assessing mental workload remains unclear. This study was to investigate the potential use of convolutional neural networks (CNNs) based on functional near-infrared spectroscopy (fNIRS)-derived signals to classify mental workload in individuals with mild cognitive impairment.

Methods

Spatial images by constructing a statistical activation map from the prefrontal activity of 120 subjects with MCI performing three difficulty levels of the N-back task (0, 1, and 2-back) were used for CNNs. The CNNs were evaluated using a 5 and 10-fold cross-validation method.

Results

As the difficulty level of the N-back task increased, the accuracy decreased and prefrontal activity increased. In addition, there was a significant difference in the accuracy and prefrontal activity across the three levels (p’s < 0.05). The accuracy of the CNNs based on fNIRS-derived spatial images evaluated by 5 and 10-fold cross-validation in classifying the difficulty levels ranged from 0.83 to 0.96.

Conclusion

fNIRS could also be a promising tool for measuring mental workload in older adults with MCI despite their cognitive decline. In addition, this study demonstrated the feasibility of the classification performance of the CNNs based on fNIRS-derived signals from the prefrontal cortex.

Morris CH, Leung YK. Pilot mental workload: how well do pilots really perform? Ergonomics. 2006;49:1581–96. https://doi.org/10.1080/00140130600857987.CrossRefPubMed

Causse M, Chua Z, Pysakhovich V, Camp ND, Matton N. Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS. Sci Rep. 2017;7:5222. https://doi.org/10.1038/s41598-017-05378-x.CrossRefPubMedPubMedCentral

Causse M, Fabre E, Giraudet L, Gonzalez M, Peysakhovich V. EEG/ERP as a measure of mental workload in a simple piloting task. Procedia Manuf. 2015;3:5230–6. https://doi.org/10.1016/j.promfg.2015.07.594.CrossRef

Tomasi D, Ernst T, Caparelli EC, Chang L. Common deactivation patterns during working memory and visual attention tasks: an intra-subject fMRI study at 4 Tesla. Hum Brain Mapp. 2006;27:694–705. https://doi.org/10.1002/hbm.20211.CrossRefPubMedPubMedCentral

Seok D, Lee S, Kim M, Cho J, Kim C. Motion artifact removal techniques for wearable EEG and PPG sensor systems. Front Electron. 2021;2:685513. https://doi.org/10.3389/felec.2021.685513.CrossRef

Ayaz H, Shewokis PA, Bunce S, Izzetoglu K, Willems B, Onaral B. Optical brain monitoring for operator training and mental workload assessment. NeuroImage. 2012;59:36–47. https://doi.org/10.1016/j.neuroimage.2011.06.023.CrossRefPubMed

Ehlis AC, Bahne CG, Jacob CP, Herrmann MJ, Fallgatter AJ. Reduced lateral prefrontal activation in adult patients with attention-deficit/hyperactivity disorder (ADHD) during a working memory task: a functional near-infrared spectroscopy (fNIRS) study. J Psychiatr Res. 2008;42:1060–7. https://doi.org/10.1016/j.jpsychires.2007.11.011.CrossRefPubMed

Kwee IL, Nakada T. Dorsolateral prefrontal lobe activation declines significantly with age functional NIRS study. J Neurol. 2003;250:525–9. https://doi.org/10.1007/s00415-003-1028-x.CrossRefPubMed

Fishburn FA, Norr ME, Medvedev AV, Vaidya CJ. Sensitivity of fNIRS to cognitive state and load. Front Hum Neurosci. 2014;8:76. https://doi.org/10.3389/fnhum.2014.00076.CrossRefPubMedPubMedCentral

10.

Reuter-Lorenz PA, Cappell KA. Neurocognitive aging and the compensation hypothesis. Curr Dir Psychol Sci. 2008;17:177–82. https://doi.org/10.1111/j.1467-8721.2008.0057.CrossRef

11.

Park JH. Can the fNIRS-derived neural biomarker better discriminate mild cognitive impairment than a neuropsychological screening test? Front Aging Neurosci. 2023;15:1137283. https://doi.org/10.3389/fnagi.2023.1137283.CrossRefPubMedPubMedCentral

12.

Yang D, Huang R, Yoo SH, Shin MJ, Yoon JA, Shin YI, Hong KS. Detection of mild cognitive impairment using convolutional neural network: temporal-feature maps of functional near-infrared spectroscopy. Front Aging Neurosci. 2020;12:141. https://doi.org/10.3389/fnagi.2020.00141.CrossRefPubMedPubMedCentral

13.

Liu Y, Ayaz H, Shewokis PA. Multisubject learning for mental workload classification using concurrent EEG, fNIRS, and physiological measures. Front Hum Neurosci. 2017;11:389. https://doi.org/10.3389/fnhum.2017.00389.CrossRefPubMedPubMedCentral

14.

Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004;256:183–94. https://doi.org/10.1111/j.1365-2796.2004.01388.x.CrossRefPubMed

15.

Park JH. Machine-learning algorithms based on screening tests for mild cognitive impairment. Am J Alzheimers Dis Other Demen. 2020;35:1533317520927163. https://doi.org/10.1177/1533317520927163.CrossRefPubMed

16.

Villringer A, Planck J, Hock C, Schleinkofer L, Dirnagl U. Near infrared spectroscopy (NIRS): a new tool to study hemodynamic changes during activation of brain function in human adults. Neurosci Lett. 1993;154:101–4. https://doi.org/10.1016/0304-3940(93)90181-j.CrossRefPubMed

17.

Maidan I, Bernad-Elazari H, Giladi N, Hausdorff JM, Mirelman A. When is higher level cognitive control needed for locomotor tasks among patients with Parkinson’s Disease? Brain Topogr. 2017;30:531–8. https://doi.org/10.1007/s10548-017-0564-0.CrossRefPubMed

18.

Scholkmann F, Kleiser S, Metz AJ, Zimmermann R, Pavia JM, Wolf U, Wolf M. A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology. NeuroImage. 2014;85:6–27. https://doi.org/10.1016/j.neuroimage.2013.05.004.CrossRefPubMed

19.

Hu XS, Arredondo MM, Gomba M, Confer N, DaSilva AF, Johnson TD, et al. Comparison of motion correction techniques applied to functional near-infrared spectroscopy data from children. J Biomed Opt. 2015;20:126003. https://doi.org/10.1117/1.JBO.20.12.126003.CrossRefPubMedPubMedCentral

20.

Molavi B, Dumont GA. Wavelet-based motion artifact removal for functional near-infrared spectroscopy. Physiol Meas. 2012;33:259–70. https://doi.org/10.1088/0967-3334/33/2/259.CrossRefPubMed

21.

Miyai I, Tanabe HC, Sase I, Eda H, Oda I, Konish I, et al. Cortical mapping gait in humans: a near-infrared spectroscopic topography study. NeuroImage. 2001;14:1186–92. https://doi.org/10.1006/nimg.2001.0905.CrossRefPubMed

22.

Li R, Nguyen T, Potter T, Zhang Y. Dynamic cortical connectivity alterations associated with Alzheimer’s Disease: an EEG and fNIRS integration study. NeuroImage. 2019;21:101622. https://doi.org/10.1016/j.nicl.2018.101622.CrossRefPubMed

23.

Saadati M, Nelson J, Ayaz H. Mental workload classification from spatial representation of fNIRS recordings using convolutional neural networks. In: 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP). 2019. p. 1–6.

24.

Lee JY, Lee DW, Cho SJ, Na DL, Jeon HJ, Kim SK, et al. Brief screening for mild cognitive impairment in elderly outpatient clinic: validation of the Korean version of the Montreal cognitive assessment. J Geriatr Psychiatry Neurol. 2008;21:104–10. https://doi.org/10.1177/0891988708316855.CrossRefPubMed

25.

Herff C, Heger D, Fortmann O, Hennrich J, Putze F, Schultz T. Mental workload during n-back task-quantified in the prefrontal cortex using fNIRS. Front Hum Neurosci. 2013;7:935–40. https://doi.org/10.3389/fnhum.2013.00935.CrossRefPubMed

26.

Mandrick K, Peysakhovich V, Remy F, Lepron E, Causse M. Neural and psychophysiological correlates of human performance under stress and high mental workload. Bio Psychol. 2016;121:62–73. https://doi.org/10.1016/j.biopsycho.2016.10.002.CrossRef

27.

Prat CS, Keller TA, Just MA. Individual differences in sentence comprehension: a functional magnetic resonance imaging investigation of syntactic and lexical processing demands. J Cogn Neurosci. 2007;19:1950–63. https://doi.org/10.1162/jocn.2007.19.12.1950.CrossRefPubMedPubMedCentral

28.

Helton WS, Warm JS, Tripp LD, Matthews G, Parasuraman R, Hancock PA. Cerebral lateralization of vigilance: a function of task difficulty. Neuropsychologia. 2010;48:1683–8. https://doi.org/10.1016/j.neuropsychologia.2010.02.014.CrossRefPubMed

29.

Midha S, Maior HA, Wilson ML, Sharples S. Measuring mental workload variations in office work tasks using fNIRS. Int J Hum Comput Stud. 2021;147:102580. https://doi.org/10.1016/j.ijhcs.2020.102580.CrossRef

30.

Yang D, Hong KS, Yoo SH, Kim CS. Evaluation of neural degeneration biomarkers in the prefrontal cortex for early identification of patients with mild cognitive impairment: an fNIRS study. Front Hum Neurosci. 2019;13:317. https://doi.org/10.3389/fnhum.2019.00317.CrossRefPubMedPubMedCentral

31.

Kwon J, Im CH. Subject-independent functional Near-Infrared spectroscopy-based brain–computer interfaces based on convolutional neural networks. Front Hum Neurosci. 2021;15:646915. https://doi.org/10.3389/fnhum.2021.646915.CrossRefPubMedPubMedCentral

32.

Naseer N, Hong MJ, Hong KS. Online binary decision decoding using functional near-infrared spectroscopy for the development of brain–computer interface. Exp Brain Res. 2014;232:555–64. https://doi.org/10.1007/s00221-013-3764-1.CrossRefPubMed

33.

Naseer N, Hong KS. Classification of functional near-infrared spectroscopy signals corresponding to the right-and left-wrist motor imagery for development of a brain–computer interface. Neurosci Lett. 2013;553:84–9. https://doi.org/10.1016/j.neulet.2013.08.021.CrossRefPubMed

34.

Jeun YJ, Nam Y, Lee SA, Park JH. Effects of Personalized Cognitive Training with the machine learning algorithm on neural efficiency in healthy younger adults. Int J Environ Res Public Health. 2022;19:13044. https://doi.org/10.3390/ijerph192013044.CrossRefPubMedPubMedCentral

35.

Tian J, Wang J, Quan W, Lv B, Liu J, Hobbie C, et al. The functional near-infrared spectroscopy in the diagnosis of schizophrenia. Eur J Psychiatry. 2019;33:97–103. https://doi.org/10.1016/j.ejpsy.2019.05.001.CrossRef

36.

Grabner RH, Fink A, Stipacek A, Neuper C, Neubauer AC. Intelligence and working memory systems: evidence of neural efficiency in alpha band ERD. Brain Res Cogn Brain Res. 2004;20:212–25. https://doi.org/10.1016/j.cogbrainres.2004.02.010.CrossRefPubMed

37.

Grabner RH, Neubauer AC, Stern E. Superior performance and neural efficiency: the impact of intelligence and expertise. Brain Res Bull. 2006;69:422–39. https://doi.org/10.1016/j.brainresbull.2006.02.009.CrossRefPubMed

38.

Matsuda G, Hiraki K. Sustained decrease in oxygenated hemoglobin during video games in the dorsal prefrontal cortex: a NIRS study of children. NeuroImage. 2006;29:706–11. https://doi.org/10.1016/j.neuroimage.2005.08.019.CrossRefPubMed

Title: Mental workload classification using convolutional neural networks based on fNIRS-derived prefrontal activity
Author: Jin-Hyuck Park
Publication date: 01-12-2023
Publisher: BioMed Central
Keyword: Mild Neurocognitive Disorder
Published in: BMC Neurology / Issue 1/2023
Electronic ISSN: 1471-2377
DOI: https://doi.org/10.1186/s12883-023-03504-z

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Mental workload classification using convolutional neural networks based on fNIRS-derived prefrontal activity

Abstract

Background

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2023

A case of spontaneous basal ganglia hemorrhage with contralateral extension utilizing the canal of Gratiolet

Activate & fire: a feasibility study in combining acoustic stimulation and continuous theta burst stimulation in chronic tinnitus

Identification of LDLR mutation in cerebral venous sinus thrombosis co-existing with dural arteriovenous fistulas: a case report

Early clinical experience with eptinezumab: results of a retrospective observational study of patient response in the United States

Abnormal alterations of regional spontaneous neuronal activity and functional connectivity in insomnia patients with difficulty falling asleep: a resting-state fMRI study

The value of elevated cerebrospinal fluid lactate concentrations in post-neurosurgical bacterial meningitis