Top

Published in:

Open Access 01-01-2013 | Original Paper

Navigation of a Telepresence Robot via Covert Visuospatial Attention and Real-Time fMRI

Authors: Patrik Andersson, Josien P. W. Pluim, Max A. Viergever, Nick F. Ramsey

Published in: Brain Topography | Issue 1/2013

Abstract

Brain–computer interfaces (BCIs) allow people with severe neurological impairment and without ability to control their muscles to regain some control over their environment. The BCI user performs a mental task to regulate brain activity, which is measured and translated into commands controlling some external device. We here show that healthy participants are capable of navigating a robot by covertly shifting their visuospatial attention. Covert Visuospatial Attention (COVISA) constitutes a very intuitive brain function for spatial navigation and does not depend on presented stimuli or on eye movements. Our robot is equipped with motors and a camera that sends visual feedback to the user who can navigate it from a remote location. We used an ultrahigh field MRI scanner (7 Tesla) to obtain fMRI signals that were decoded in real time using a support vector machine. Four healthy subjects with virtually no training succeeded in navigating the robot to at least three of four target locations. Our results thus show that with COVISA BCI, realtime robot navigation can be achieved. Since the magnitude of the fMRI signal has been shown to correlate well with the magnitude of spectral power changes in the gamma frequency band in signals measured by intracranial electrodes, the COVISA concept may in future translate to intracranial application in severely paralyzed people.

Available only for authorised users

Allison BZ, McFarland DJ, Schalk G, Zheng SD, Jackson MM, Wolpaw JR (2008) Towards an independent brain–computer interface using steady state visual evoked potentials. Clin Neurophysiol 119(2):399–408PubMedCrossRef

Andersson P, Pluim JPW, Siero JCW, Klein S, Viergever MA, Ramsey NF (2011) Real-time decoding of brain responses to visuospatial attention using 7T fMRI. PLoS ONE 6(11):e27638PubMedCrossRef

Andersson P, Ramsey NF, Raemaekers M, Viergever MA, Pluim JPW (2012) Real-time decoding of direction of covert visuospatial attention. J Neur Eng 9(4):045004CrossRef

Bagarinao E, Matsuo K, Nakai T, Sato S (2003) Estimation of general linear model coefficients for real-time application. Neuroimage 19(2):422–429PubMedCrossRef

Bianchi L, Sami S, Hillebrand A, Fawcett IP, Quitadamo LR, Seri S (2010) Which physiological components are more suitable for visual ERP based brain–computer interface? A preliminary MEG/EEG study. Brain Topogr 23(2):180–5PubMedCrossRef

Brefczynski JA, DeYoe EA (1999) A physiological correlate of the ‘spotlight’ of visual attention. Nat Neurosci 2(4):370–374PubMedCrossRef

Brefczynski-Lewis JA, Datta R, Lewis JW, DeYoe EA (2009) The topography of visuospatial attention as revealed by a novel visual field mapping technique. J Cogn Neurosci 21(7):1447–1460PubMedCrossRef

Brunner P, Joshi S, Briskin S, Wolpaw JR, Bischof H, G S (2010) Does the ‘P300’ speller depend on eye gaze? J Neural Eng 7(5):056,013CrossRef

Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM T Int Syst Tech 2(3):1–27CrossRef

Datta R, DeYoe EA (2009) I know where you are secretly attending! The topography of human visual attention revealed with fMRI. Vis Res 49(10):1037–1044PubMedCrossRef

De Martino F, Valente G, Staeren N, Ashburner J, Goebel R, Formisano E (2008) Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns. Neuroimage 43(1):44–58PubMedCrossRef

Dougherty RF, Koch VM, Brewer AA, Fischer B, Modersitzki J, Wandell BA (2003) Visual field representations and locations of visual areas V1/2/3 in human visual cortex. J Vis 3(10)

Farwell LA, Donchin E (1988) Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol 70(6):510–523PubMedCrossRef

Gunduz A, Brunner P, Daitch A, Leuthardt EC, Ritaccio AL, Pesaran B, Schalk G (2011) Neural correlates of visual spatial attention in electrocorticographic (ECoG) signals in humans. Front Hum Neurosci 5(89)

Heinemann L, Kleinschmidt A, Müller NG (2009) Exploring BOLD changes during spatial attention in non-stimulated visual cortex. PLoS ONE 4(5):e5560PubMedCrossRef

Hermes D, Miller KJ, Vansteensel MJ, Aarnoutse EJ, Leijten FSS, Ramsey NF (2012) Neurophysiologic correlates of fMRI in human motor cortex. Hum Brain Map 33(7):1689–1699CrossRef

Kastner S, Pinsk MA, De Weerd P, Desimone R, Ungerleider LG (1999) Increased activity in human visual cortex during directed attention in the absence of visual stimulation. Neuron 22(4):751–761PubMedCrossRef

Kelly SP, Lalor E, Reilly RB, Foxe JJ (2005) Independent brain computer interface control using visual spatial attention-dependent modulations of parieto-occipital alpha. In: 2nd International IEEE EMBS conference on neural engineering, pp 667–670

Klein S, Staring M, Pluim JPW (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines. IEEE Trans Image Process 16(12):2879–2890PubMedCrossRef

Lachaux JP, Fonlupt P, Kahane P, Minotti L, Hoffmann D, Bertrand O, Baciu M (2007) Relationship between task-related gamma oscillations and BOLD signal: New insights from combined fMRI and intracranial EEG. Hum Brain Mapp 28(12):1368–1375PubMedCrossRef

LaConte S, Strother S, Cherkassky V, Anderson J, Hu X (2005) Support vector machines for temporal classification of block design fMRI data. Neuroimage 26(2):317–329PubMedCrossRef

LaConte SM, Peltier SJ, Hu XP (2007) Real-time fMRI using brain-state classification. Hum Brain Mapp 28(10):1033–1044PubMedCrossRef

Leuthardt EC, Schalk G, Roland J, Rouse A, Moran DW (2009) Evolution of brain–computer interfaces: going beyond classic motor physiology. Neurosurg Focus 27(1):E4PubMedCrossRef

Matthews F, Pearlmutter BA, Ward TE, Soraghan CASC, Markham CAMC (2008) Hemodynamics for brain–computer interfaces. IEEE Signal Process Mag 25(1):87–94CrossRef

Moench T, Hollmann M, Grzeschik R, Mueller C, Luetzkendorf R, Baecke S, Luchtmann M, Wagegg D, Bernarding J (2008) Real-time classification of activated brain areas for fMRI-based human–brain-interfaces. SPIE 6916:69,161R–10

Munneke J, Heslenfeld DJ, Theeuwes J (2008) Directing attention to a location in space results in retinotopic activation in primary visual cortex. Brain Res Brain Res Rev 1222:184–191PubMedCrossRef

Perry RJ, Zeki S (2000) The neurology of saccades and covert shifts in spatial attention. Brain Behav Evol 123(11):2273–2288PubMedCrossRef

Shishkin SL, Ganin IP, Basyul IA, Zhigalov AY, Kaplan AY (2009) N1 wave in the P300 BCI is not sensitive to the physical characteristics of stimuli. J Integr Neurosci 8(4):471–85PubMedCrossRef

Siero JCW, Petridou N, Hoogduin H, Luijten PR, Ramsey NF (2011) Cortical depth-dependent temporal dynamics of the BOLD response in the human brain. J Cereb Blood Flow Metab 31(10):1999–2008PubMedCrossRef

Siman-Tov T, Mendelsohn A, Schonberg T, Avidan G, Podlipsky I, Pessoa L, Gadoth N, Ungerleider LG, Hendler T (2007) Bihemispheric leftward bias in a visuospatial attention-related network. J Neurosci 27(42):11,271–11,278CrossRef

Sitaram R, Caria A, Veit R, Gaber T, Rota G, Kuebler A, Birbaumer N (2007) fMRI brain–computer interface: a tool for neuroscientific research and treatment. Comp Intell Neurosci 2007:25487

Sitaram R, Weiskopf N, Caria A, Veit R, Erb M, Birbaumer N (2008) fMRI brain–computer interfaces. IEEE Signal Proc Mag 25(1):95–106CrossRef

Sitaram R, Lee S, Ruiz S, Rana M, Veit R, Birbaumer N (2011) Real-time support vector classification and feedback of multiple emotional brain states. Neuroimage 56(2):753–65PubMedCrossRef

Sorger B, Reithler J, Dahmen B, Goebel R (2012) A real-time fMRI-based spelling device immediately enabling robust motor-independent communication. Curr Biol 22(14):1333–1338PubMedCrossRef

Tarvainen MP, Ranta-aho PO, Karjalainen PA (2002) An advanced detrending method with application to HRV analysis. IEEE Trans Biomed Eng 49(2):172–175PubMedCrossRef

Treder MS, Blankertz B (2010) (C)overt attention and visual speller design in an ERP-based brain–computer interface. Behav Brain Funct 6(1):28PubMedCrossRef

Treder MS, Schmidt NM, Blankertz B (2011b) Gaze-independent brain–computer interfaces based on covert attention and feature attention. J Neural Eng 8(6)

Treder M, Bahramisharif A, Schmidt N, van Gerven M, Blankertz B (2011a) Brain–computer interfacing using modulations of alpha activity induced by covert shifts of attention. J NeuroEng Rehabil 8(1):24PubMedCrossRef

Vansteensel MJ, Hermes D, Aarnoutse EJ, Bleichner MG, Schalk G, van Rijen PC, Leijten FSS, Ramsey NF (2010) Brain–computer interfacing based on cognitive control. Ann Neurol 67(6):809–816PubMed

van Gerven M, Jensen O (2009) Attention modulations of posterior alpha as a control signal for two-dimensional brain–computer interfaces. J Neurosci Methods 179(1):78–84PubMedCrossRef

van Gerven M, Bahramisharif A, Heskes T, Jensen O (2009) Selecting features for BCI control based on a covert spatial attention paradigm. Neural Netw 22(9):1271–1277PubMedCrossRef

Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM (2002) Brain–computer interfaces for communication and control. Clin Neurophysiol 113(6):767–791PubMedCrossRef

Yamamoto H, Fukunaga M, Takahashi S, Mano H, Tanaka C, Umeda M, Ejima Y (2012) Inconsistency and uncertainty of the human visual area loci following surface-based registration: probability and entropy maps. Hum Brain Map 33(1):121–129CrossRef

Yoo SS, Fairneny T, Chen NK, Choo SE, Panych LP, Park H, Lee SY, Jolesz FA (2004) Brain–computer interface using fMRI: spatial navigation by thoughts. Neuroreport 15(10):1591–1595PubMedCrossRef

Title: Navigation of a Telepresence Robot via Covert Visuospatial Attention and Real-Time fMRI
Authors: Patrik Andersson
Josien P. W. Pluim
Max A. Viergever
Nick F. Ramsey
Publication date: 01-01-2013
Publisher: Springer US
Published in: Brain Topography / Issue 1/2013
Print ISSN: 0896-0267
Electronic ISSN: 1573-6792
DOI: https://doi.org/10.1007/s10548-012-0252-z

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Navigation of a Telepresence Robot via Covert Visuospatial Attention and Real-Time fMRI

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 1/2013

Regional Magnetic Resonance Imaging Measures for Multivariate Analysis in Alzheimer’s Disease and Mild Cognitive Impairment

Electrophysiological Correlation Patterns of Resting State Networks in Single Subjects: A Combined EEG–fMRI Study

Neuronal Networks in West Syndrome as Revealed by Source Analysis and Renormalized Partial Directed Coherence

A Novel Method for Reducing the Effect of Tonic Muscle Activity on the Gamma Band of the Scalp EEG

Musicianship Boosts Perceptual Learning of Pseudoword-Chimeras: An Electrophysiological Approach

A Short-Term Scan–Rescan Reliability Test Measuring Brain Tissue and Subcortical Hyperintensity Volumetrics Obtained Using the Lesion Explorer Structural MRI Processing Pipeline