Skip to main content
Top
Published in: Journal of NeuroEngineering and Rehabilitation 1/2019

Open Access 01-12-2019 | Electroencephalography | Research

User activity recognition system to improve the performance of environmental control interfaces: a pilot study with patients

Authors: Arturo Bertomeu-Motos, Santiago Ezquerro, Juan A. Barios, Luis D. Lledó, Sergio Domingo, Marius Nann, Suzanne Martin, Surjo R. Soekadar, Nicolas Garcia-Aracil

Published in: Journal of NeuroEngineering and Rehabilitation | Issue 1/2019

Login to get access

Abstract

Background

Assistive technologies aim to increase quality of life, reduce dependence on care giver and on the long term care system. Several studies have demonstrated the effectiveness in the use of assistive technology for environment control and communication systems. The progress of brain-computer interfaces (BCI) research together with exoskeleton enable a person with motor impairment to interact with new elements in the environment. This paper aims to evaluate the environment control interface (ECI) developed under the AIDE project conditions, a multimodal interface able to analyze and extract relevant information from the environments as well as from the identification of residual abilities, behaviors, and intentions of the user.

Methods

This study evaluated the ECI in a simulated scenario using a two screen layout: one with the ECI and the other with a simulated home environment, developed for this purpose. The sensorimotor rhythms and the horizontal oculoversion, acquired through BCI2000, a multipurpose standard BCI platform, were used to online control the ECI after the user training and system calibration. Eight subjects with different neurological diseases and spinal cord injury participated in this study. The subjects performed simulated activities of daily living (ADLs), i.e. actions in the simulated environment as drink, switch on a lamp or raise the bed head, during ten minutes in two different modes, AIDE mode, using a prediction model, to recognize the user intention facilitating the scan, and Manual mode, without a prediction model.

Results

The results show that the mean task time spent in the AIDE mode was less than in the Manual, i.e the users were able to perform more tasks in the AIDE mode during the same time. The results showed a statistically significant differences with p<0.001. Regarding the steps, i.e the number of abstraction levels crossed in the ECI to perform an ADL, the users performed one step in the 90% of the tasks using the AIDE mode and three steps, at least, were necessary in the Manual mode. The user’s intention prediction was performed through conditional random fields (CRF), with a global accuracy about 87%.

Conclusions

The environment analysis and the identification of the user’s behaviors can be used to predict the user intention opening a new paradigm in the design of the ECIs. Although the developed ECI was tested only in a simulated home environment, it can be easily adapted to a real environment increasing the user independence at home.
Literature
2.
go back to reference Winstein CJ, Stein J, Arena R, Bates B, Cherney LR, Cramer SC, Deruyter F, Eng JJ, Fisher B, Harvey RL, et al.Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the american heart association/american stroke association. Stroke. 2016; 47(6):98–169.CrossRef Winstein CJ, Stein J, Arena R, Bates B, Cherney LR, Cramer SC, Deruyter F, Eng JJ, Fisher B, Harvey RL, et al.Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the american heart association/american stroke association. Stroke. 2016; 47(6):98–169.CrossRef
3.
go back to reference Sellers EW, Vaughan TM, Wolpaw JR. A brain-computer interface for long-term independent home use. Amyotroph Lateral Scler. 2010; 11(5):449–55.CrossRef Sellers EW, Vaughan TM, Wolpaw JR. A brain-computer interface for long-term independent home use. Amyotroph Lateral Scler. 2010; 11(5):449–55.CrossRef
4.
go back to reference Ball MM, Perkins MM, Whittington FJ, Hollingsworth C, King SV, Combs BL. Independence in assisted living. J Aging Stud. 2004; 18(4):467–83.CrossRef Ball MM, Perkins MM, Whittington FJ, Hollingsworth C, King SV, Combs BL. Independence in assisted living. J Aging Stud. 2004; 18(4):467–83.CrossRef
5.
go back to reference Ball MM, Perkins MM, Whittington FJ, Connell BR, Hollingsworth C, King SV, Elrod CL, Combs BL. Managing decline in assisted living: The key to aging in place. J Gerontol B Psychol Sci Soc Sci. 2004; 59(4):202–12.CrossRef Ball MM, Perkins MM, Whittington FJ, Connell BR, Hollingsworth C, King SV, Elrod CL, Combs BL. Managing decline in assisted living: The key to aging in place. J Gerontol B Psychol Sci Soc Sci. 2004; 59(4):202–12.CrossRef
6.
go back to reference Struijk LNA, Egsgaard LL, Lontis R, Gaihede M, Bentsen B. Wireless intraoral tongue control of an assistive robotic arm for individuals with tetraplegia. J Neuroengineering Rehabil. 2017; 14(1):110.CrossRef Struijk LNA, Egsgaard LL, Lontis R, Gaihede M, Bentsen B. Wireless intraoral tongue control of an assistive robotic arm for individuals with tetraplegia. J Neuroengineering Rehabil. 2017; 14(1):110.CrossRef
7.
go back to reference Brandt Å, Samuelsson K, Töytäri O, Salminen A-L. Activity and participation, quality of life and user satisfaction outcomes of environmental control systems and smart home technology: a systematic review. Disabil Rehabil Assist Technol. 2011; 6(3):189–206.CrossRef Brandt Å, Samuelsson K, Töytäri O, Salminen A-L. Activity and participation, quality of life and user satisfaction outcomes of environmental control systems and smart home technology: a systematic review. Disabil Rehabil Assist Technol. 2011; 6(3):189–206.CrossRef
8.
go back to reference Agree EM, Freedman VA, Cornman JC, Wolf DA, Marcotte JE. Reconsidering substitution in long-term care: when does assistive technology take the place of personal care?J Gerontol B Psychol Sci Soc Sci. 2005; 60(5):272–80.CrossRef Agree EM, Freedman VA, Cornman JC, Wolf DA, Marcotte JE. Reconsidering substitution in long-term care: when does assistive technology take the place of personal care?J Gerontol B Psychol Sci Soc Sci. 2005; 60(5):272–80.CrossRef
9.
go back to reference O’Neill B, Moran K, Gillespie A. Scaffolding rehabilitation behaviour using a voice-mediated assistive technology for cognition. Neuropsychol Rehabil. 2010; 20(4):509–27.CrossRef O’Neill B, Moran K, Gillespie A. Scaffolding rehabilitation behaviour using a voice-mediated assistive technology for cognition. Neuropsychol Rehabil. 2010; 20(4):509–27.CrossRef
12.
go back to reference Biswas P, Langdon P. A new input system for disabled users involving eye gaze tracker and scanning interface. J Assist Technol. 2011; 5(2):58–66.CrossRef Biswas P, Langdon P. A new input system for disabled users involving eye gaze tracker and scanning interface. J Assist Technol. 2011; 5(2):58–66.CrossRef
13.
go back to reference Miralles F, Vargiu E, Rafael-Palou X, Sola M, Dauwalder S, Guger C, Hintermuller C, Espinosa A, Lowish H, Martin S, Armstrong E, Daly J. Brain?computer interfaces on track to home: Results of the evaluation at disabled end-users? homes and lessons learnt. Front ICT. 2015; 2:25. https://doi.org/10.3389/fict.2015.00025.CrossRef Miralles F, Vargiu E, Rafael-Palou X, Sola M, Dauwalder S, Guger C, Hintermuller C, Espinosa A, Lowish H, Martin S, Armstrong E, Daly J. Brain?computer interfaces on track to home: Results of the evaluation at disabled end-users? homes and lessons learnt. Front ICT. 2015; 2:25. https://​doi.​org/​10.​3389/​fict.​2015.​00025.CrossRef
14.
go back to reference Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain–computer interfaces for communication and control. Clin Neurophysiol. 2002; 113(6):767–91.CrossRef Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain–computer interfaces for communication and control. Clin Neurophysiol. 2002; 113(6):767–91.CrossRef
15.
go back to reference Cincotti F, Mattia D, Aloise F, Bufalari S, Schalk G, Oriolo G, Cherubini A, Marciani MG, Babiloni F. Non-invasive brain–computer interface system: towards its application as assistive technology. Brain Res Bull. 2008; 75(6):796–803.CrossRef Cincotti F, Mattia D, Aloise F, Bufalari S, Schalk G, Oriolo G, Cherubini A, Marciani MG, Babiloni F. Non-invasive brain–computer interface system: towards its application as assistive technology. Brain Res Bull. 2008; 75(6):796–803.CrossRef
17.
go back to reference Soekadar SR, Witkowski M, Mellinger J, Ramos A, Birbaumer N, Cohen LG. Erd-based online brain-machine interfaces (bmi) in the context of neurorehabilitation: optimizing bmi learning and performance.IEEE Trans Neural Syst Rehabil Eng: Publ IEEE Eng Med Biol Soc. 2011; 19(5):542–9.CrossRef Soekadar SR, Witkowski M, Mellinger J, Ramos A, Birbaumer N, Cohen LG. Erd-based online brain-machine interfaces (bmi) in the context of neurorehabilitation: optimizing bmi learning and performance.IEEE Trans Neural Syst Rehabil Eng: Publ IEEE Eng Med Biol Soc. 2011; 19(5):542–9.CrossRef
18.
go back to reference Lafferty J, McCallum A, Pereira FC. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML ’01). San Francisco: Morgan Kaufmann Publishers Inc.: 2001. p. 282–289. http://portal.acm.org/citation.cfm?id=655813. Lafferty J, McCallum A, Pereira FC. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML ’01). San Francisco: Morgan Kaufmann Publishers Inc.: 2001. p. 282–289. http://​portal.​acm.​org/​citation.​cfm?​id=​655813.
19.
go back to reference Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR. Bci2000: a general-purpose brain-computer interface (bci) system. IEEE Trans Biomed Eng. 2004; 51(6):1034–43.CrossRef Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR. Bci2000: a general-purpose brain-computer interface (bci) system. IEEE Trans Biomed Eng. 2004; 51(6):1034–43.CrossRef
20.
go back to reference Barios JA, Ezquerro S, Bertomeu-Motos A, Nann M, Badesa FJ, Fernandez E, Soekada SR, Garcia-Aracil N. Synchronization of slow cortical rhythms during motor imagery-based brain-machine interface control. Int J Neural Syst. 2018. in press. Barios JA, Ezquerro S, Bertomeu-Motos A, Nann M, Badesa FJ, Fernandez E, Soekada SR, Garcia-Aracil N. Synchronization of slow cortical rhythms during motor imagery-based brain-machine interface control. Int J Neural Syst. 2018. in press.
21.
go back to reference Rabiner LR. A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE. 1989; 77(2):257–86.CrossRef Rabiner LR. A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE. 1989; 77(2):257–86.CrossRef
22.
go back to reference Quinn TJ, Langhorne P, Stott DJ. Barthel index for stroke trials: development, properties, and application. Stroke. 2011; 42(4):1146–51.CrossRef Quinn TJ, Langhorne P, Stott DJ. Barthel index for stroke trials: development, properties, and application. Stroke. 2011; 42(4):1146–51.CrossRef
Metadata
Title
User activity recognition system to improve the performance of environmental control interfaces: a pilot study with patients
Authors
Arturo Bertomeu-Motos
Santiago Ezquerro
Juan A. Barios
Luis D. Lledó
Sergio Domingo
Marius Nann
Suzanne Martin
Surjo R. Soekadar
Nicolas Garcia-Aracil
Publication date
01-12-2019
Publisher
BioMed Central
Published in
Journal of NeuroEngineering and Rehabilitation / Issue 1/2019
Electronic ISSN: 1743-0003
DOI
https://doi.org/10.1186/s12984-018-0477-5

Other articles of this Issue 1/2019

Journal of NeuroEngineering and Rehabilitation 1/2019 Go to the issue