Top

Published in:

01-10-2005 | Research Article

Is the auditory sensory memory sensitive to visual information?

Authors: Julien Besle, Alexandra Fort, Marie-Hélène Giard

Published in: Experimental Brain Research | Issue 3-4/2005

Abstract

The mismatch negativity (MMN) component of auditory event-related brain potentials can be used as a probe to study the representation of sounds in auditory sensory memory (ASM). Yet it has been shown that an auditory MMN can also be elicited by an illusory auditory deviance induced by visual changes. This suggests that some visual information may be encoded in ASM and is accessible to the auditory MMN process. It is not known, however, whether visual information affects ASM representation for any audiovisual event or whether this phenomenon is limited to specific domains in which strong audiovisual illusions occur. To highlight this issue, we have compared the topographies of MMNs elicited by non-speech audiovisual stimuli deviating from audiovisual standards on the visual, the auditory, or both dimensions. Contrary to what occurs with audiovisual illusions, each unimodal deviant elicited sensory-specific MMNs, and the MMN to audiovisual deviants included both sensory components. The visual MMN was, however, different from a genuine visual MMN obtained in a visual-only control oddball paradigm, suggesting that auditory and visual information interacts before the MMN process occurs. Furthermore, the MMN to audiovisual deviants was significantly different from the sum of the two sensory-specific MMNs, showing that the processes of visual and auditory change detection are not completely independent.

As, on the one hand, the topography of the auditory MMN is well known and on the other hand, it would have needlessly lengthened the recording session, we chose not to conduct an auditory oddball paradigm

Alain C, Woods DL, Knight RT (1998) A distributed cortical network for auditory sensory memory in humans. Brain Res 812:23–37PubMed

Bental E, Dafny N, Feldman S (1968) Convergence of auditory and visual stimuli on single cells in the primary visual cortex of unanesthetized unrestrained cats. Exp Neurol 20:341–351PubMed

Bertelson P, Aschersleben G (1998) Automatic visual bias of perceived auditory location. Psychon Bull Rev 5:482–489

Berti S, Schroger E (2004) Distraction effects in vision: behavioral and event-related potential indices. Neuroreport 15:665–669PubMed

Besle J, Fort A, Delpuech C, Giard M-H (2004) Bimodal speech: early suppressive visual effects in the human auditory cortex. Eur J Neurosci 20:2225–2234PubMed

Bruneau N, Roux S, Garreau B, Martineau J, Lelord G (1990) Cortical evoked potentials as indicators of Auditory-Visual Cross-Modal Association in young adults. Pavlov J Biol Sci 25:189–204

Cahill L, Ohl F, Scheich H (1996) Alteration of auditory cortex activity with a visual stimulus through conditioning: a 2-deoxyglucose analysis. Neurobiol Learn Mem 65:213–222PubMed

Colin C, Radeau M, Soquet A, Demolin D, Colin F, Deltenre P (2002b) Mismatch negativity evoked by the McGurk–MacDonald effect: a phonetic representation within short-term memory. Clin Neurophysiol 113:495–506PubMed

Colin C, Radeau M, Soquet A, Dachy B, Deltenre P (2002a) Electrophysiology of spatial scene analysis: the mismatch negativity (MMN) is sensitive to the ventriloquism illusion. Clin Neurophysiol 113:507–518PubMed

Colin C, Radeau M, Soquet A, Deltenre P (2004) Generalization of the generation of an MMN by illusory McGurk percepts: voiceless consonants. Clin Neurophysiol 115:1989–2000PubMed

Czigler I, Balazs L, Winkler I (2002) Memory-based detection of task-irrelevant visual changes. Psychophysiology 39:869–873PubMed

Fort A, Giard M-H (2004) Multiple electrophysiological mechanisms of audio-visual integration in human perception. In: Calvert G, Spence C, Stein B (eds) The handbook of multisensory processes. MIT Press, Cambridge

Fort A, Delpuech C, Pernier J, Giard MH (2002a) Dynamics of cortico-subcortical crossmodal operations involved in audio-visual object detection in humans. Cereb Cortex 12:1031–1039PubMed

Fort A, Delpuech C, Pernier J, Giard MH (2002b) Early auditory-visual interactions in human cortex during nonredundant target identification. Brain Res Cogn Brain Res 14:20–30PubMed

Giard MH, Peronnet F (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J Cogn Neurosci 11:473–490PubMed

Giard MH, Perrin F, Pernier J (1990) Brain generators implicated in processing of auditory stimulus deviance A topographic ERP study. Psychophysiology 27:627–640PubMed

Heslenfeld DJ (2003) Visual mismatch negativity. In: Polich J (ed) Detection of change: event-related potential and fMRI findings. Kluwer Academic Publishers, Dordrecht, pp 41–60

Kenemans JL, Jong TG, Verbaten MN (2003) Detection of visual change: mismatch or rareness?. Neuroreport 14:1239–1242PubMed

Kropotov JD, Näätänen R, Sevostianov AV, Alho K, Reinikainen K, Kropotova OV (1995) Mismatch negativity to auditory stimulus change recorded directly from the human temporal cortex. Psychophysiology 32:418–422PubMed

Lebib R, Papo D, de Bode S, Baudonniere PM (2003) Evidence of a visual-to-auditory cross-modal sensory gating phenomenon as reflected by the human P50 event-related brain potential modulation. Neurosci Lett 341:185–188PubMed

McGurk H, McDonald J (1976) Hearing lips and seeing voices. Nature 264:746–748PubMed

Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14:115–128PubMed

Möttönen R, Krause CM, Tiippana K, Sams M (2002) Processing of changes in visual speech in the human auditory cortex. Brain Res Cogn Brain Res 13:417–425PubMed

Näätänen R (1992) Attention and Brain Function. Hillsdale, NJ, USA

Näätänen R, Winkler I (1999) The concept of auditory stimulus representation in cognitive neuroscience. Psychol Bull 125:826–859PubMed

Nyman G, Alho K, laurinen P, Paavilainen P, Radil T, Rainikainen K, Sams M, Näätänen R (1990) Mismatch negativity (MMN) for sequences of auditory and visual stimuli: evidence for a mechanism specific to the auditory modality. Electroencephalogr Clin Neurophysiol 77:436–444PubMed

Pazo-Alvarez P, Cadaveira F, Amenedo E (2003) MMN in the visual modality: a review. Biol Psychol 63:199–236PubMed

Pernier J, Perrin F, Bertrand O (1988) Scalp current density fields: concept and properties. Electroencephalogr Clin Neurophysiol 69:385–389PubMed

Perrin F, Pernier J, Bertrand O, Giard M-H (1987) Mapping of scalp potentials by surface spline interpolation. Electroencephalogr Clin Neurophysiol 66:75–81PubMed

Perrin F, Pernier J, Bertrand O, Echallier JF (1989) Spherical splines for scalp potential and current density mapping. Electroencephalogr Clin Neurophysiol 72:184–187PubMed

Ritter W, Deacon D, Gomes H, Javitt DC, Vaughan HG Jr (1995) The mismatch negativity of event-related potentials as a probe of transient auditory memory: a review. Ear Hear 16:52–67PubMed

Rosenblum LD, Fowler CA (1991) Audiovisual investigation of the loudness-effort effect for speech and nonspeech events. J Exp Psychol Hum Percept Perform 17:976–985PubMed

Saldana HM, Rosenblum LD (1993) Visual influences on auditory pluck and bow judgments. Percept Psychophys 54:406–416PubMed

Sams M, Aulanko R, Hamalainen H, Hari R, Lounasmaa OV, Lu ST, Simola J (1991) Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neurosci Lett 127:141–145PubMed

Soto-Faraco S, Navarra J, Alsius A (2004) Assessing automaticity in audiovisual speech integration: evidence from the speeded classification task. Cognition 92:B13–B23PubMed

Stagg C, Hindley P, Tales A, Butler S (2004) Visual mismatch negativity: the detection of stimulus change. Neuroreport 15:659–663PubMed

Stekelenburg JJ, Vroomen J, de Gelder B (2004) Illusory sound shifts induced by the ventriloquist illusion evoke the mismatch negativity. Neurosci Lett 357:163–166PubMed

Vroomen J, Bertelson P, de Gelder B (2001) The ventriloquist effect does not depend on the direction of automatic visual attention. Percept Psychophys 63:651–659PubMed

Title: Is the auditory sensory memory sensitive to visual information?
Authors: Julien Besle
Alexandra Fort
Marie-Hélène Giard
Publication date: 01-10-2005
Publisher: Springer-Verlag
Published in: Experimental Brain Research / Issue 3-4/2005
Print ISSN: 0014-4819
Electronic ISSN: 1432-1106
DOI: https://doi.org/10.1007/s00221-005-2375-x

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Is the auditory sensory memory sensitive to visual information?

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 3-4/2005

Spatial tuning of tactile attention modulates visual processing within hemifields: an ERP investigation of crossmodal attention

Effect of posture change on tactile perception: impaired direction discrimination performance with interleaved fingers

On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies

Assessing the effect of visual and tactile distractors on the perception of auditory apparent motion

The development of a dialogue between cortex and midbrain to integrate multisensory information

Multisensory processing and oscillatory gamma responses: effects of spatial selective attention