Skip to main content
Top
Published in: Experimental Brain Research 6/2014

01-06-2014 | Mini-Review

On the ‘visual’ in ‘audio-visual integration’: a hypothesis concerning visual pathways

Authors: Philip Jaekl, Alexis Pérez-Bellido, Salvador Soto-Faraco

Published in: Experimental Brain Research | Issue 6/2014

Login to get access

Abstract

Crossmodal interaction conferring enhancement in sensory processing is nowadays widely accepted. Such benefit is often exemplified by neural response amplification reported in physiological studies conducted with animals, which parallel behavioural demonstrations of sound-driven improvement in visual tasks in humans. Yet, a good deal of controversy still surrounds the nature and interpretation of these human psychophysical studies. Here, we consider the interpretation of crossmodal enhancement findings under the light of the functional as well as anatomical specialization of magno- and parvocellular visual pathways, whose paramount relevance has been well established in visual research but often overlooked in crossmodal research. We contend that a more explicit consideration of this important visual division may resolve some current controversies and help optimize the design of future crossmodal research.
Literature
go back to reference Berson D, McIlwain J (1982) Retinal Y-cell activation of deep-layer cells in superior colliculus of the cat. J Neurophysiol 47(4):700–714PubMed Berson D, McIlwain J (1982) Retinal Y-cell activation of deep-layer cells in superior colliculus of the cat. J Neurophysiol 47(4):700–714PubMed
go back to reference Bookheimer SY, Zeffiro TA, Blaxton TA et al (1998) Regional cerebral blood flow during auditory responsive naming: evidence for cross-modality neural activation. NeuroReport 9:2409–2413PubMedCrossRef Bookheimer SY, Zeffiro TA, Blaxton TA et al (1998) Regional cerebral blood flow during auditory responsive naming: evidence for cross-modality neural activation. NeuroReport 9:2409–2413PubMedCrossRef
go back to reference Breitmeyer B (1975) Simple reaction time as a measure of the temporal response properties of transient and sustained channels. Vis Res 15:1411–1412PubMedCrossRef Breitmeyer B (1975) Simple reaction time as a measure of the temporal response properties of transient and sustained channels. Vis Res 15:1411–1412PubMedCrossRef
go back to reference Clark V, Hillyard S (1996) Spatial selective attention affects early extrastriate but not striate components of the visual evoked potential. J Cogn Neurosci 8:387–402PubMedCrossRef Clark V, Hillyard S (1996) Spatial selective attention affects early extrastriate but not striate components of the visual evoked potential. J Cogn Neurosci 8:387–402PubMedCrossRef
go back to reference Cleland BG, Levick WR, Sanderson KJ (1973) Properties of sustained and transient ganglion cells in the cat retina. J Physiol 228(3):649–680PubMedCentralPubMed Cleland BG, Levick WR, Sanderson KJ (1973) Properties of sustained and transient ganglion cells in the cat retina. J Physiol 228(3):649–680PubMedCentralPubMed
go back to reference Cleland B, Dubin MW, Levick WR (1971) Sustained and transient neurones in the cat’s retina and lateral geniculate nucleus. J Physiol 217:473–496PubMedCentralPubMed Cleland B, Dubin MW, Levick WR (1971) Sustained and transient neurones in the cat’s retina and lateral geniculate nucleus. J Physiol 217:473–496PubMedCentralPubMed
go back to reference Clemo HR, Keniston LP, Meredith MA (2012) Structural basis of multisensory processing: convergence. In: Murray MM, Wallace MT (eds) The Neural bases of multisensory processes. Frontiers in Neuroscience, chapter 1. CRC Press, Boca Raton Clemo HR, Keniston LP, Meredith MA (2012) Structural basis of multisensory processing: convergence. In: Murray MM, Wallace MT (eds) The Neural bases of multisensory processes. Frontiers in Neuroscience, chapter 1. CRC Press, Boca Raton
go back to reference Colombo M, Gross C (1994) Responses of inferior temporal cortex and hippocampal neurons during delayed matching-to-sample in monkeys (Macaca fascicularis). Behav Neurosci 108:443–455PubMedCrossRef Colombo M, Gross C (1994) Responses of inferior temporal cortex and hippocampal neurons during delayed matching-to-sample in monkeys (Macaca fascicularis). Behav Neurosci 108:443–455PubMedCrossRef
go back to reference Corneil BD, Van Wanrooij M, Munoz DP, Van Opstal AJ (2002) Auditory–visual interactions subserving goal-directed saccades in a complex scene. J Neurophysiol 88:438–454PubMed Corneil BD, Van Wanrooij M, Munoz DP, Van Opstal AJ (2002) Auditory–visual interactions subserving goal-directed saccades in a complex scene. J Neurophysiol 88:438–454PubMed
go back to reference Diederich A, Colonius H (2004) Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66:1388–1404PubMedCrossRef Diederich A, Colonius H (2004) Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66:1388–1404PubMedCrossRef
go back to reference Fain GL (2003) Sensory transduction. Sinauer Associates, Sunderland Fain GL (2003) Sensory transduction. Sinauer Associates, Sunderland
go back to reference Falchier A, Clavagnier S, Barone P, Kennedy H (2002) Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22:5749–5759PubMed Falchier A, Clavagnier S, Barone P, Kennedy H (2002) Anatomical evidence of multimodal integration in primate striate cortex. J Neurosci 22:5749–5759PubMed
go back to reference Fiebelkorn IC, Foxe JJ, Butler JS, Molholm S (2011) Auditory facilitation of visual-target detection persists regardless of retinal eccentricity and despite wide audiovisual misalignments. Exp Brain Res. doi:10.1007/s00221-011-2670-7 PubMed Fiebelkorn IC, Foxe JJ, Butler JS, Molholm S (2011) Auditory facilitation of visual-target detection persists regardless of retinal eccentricity and despite wide audiovisual misalignments. Exp Brain Res. doi:10.​1007/​s00221-011-2670-7 PubMed
go back to reference Garey LJ, Jones EG, Powell TP (1968) Interrelationships of striate and extrastriate cortex with the primary relay sites of the visual pathway. J Neurol Neurosurg Psychiatry 31:135–157PubMedCentralPubMedCrossRef Garey LJ, Jones EG, Powell TP (1968) Interrelationships of striate and extrastriate cortex with the primary relay sites of the visual pathway. J Neurol Neurosurg Psychiatry 31:135–157PubMedCentralPubMedCrossRef
go back to reference Gielen SC, Schmidt RA, Van den Heuvel PJ (1983) On the nature of intersensory facilitation of reaction time. Percept Psychophys 34:161–168PubMedCrossRef Gielen SC, Schmidt RA, Van den Heuvel PJ (1983) On the nature of intersensory facilitation of reaction time. Percept Psychophys 34:161–168PubMedCrossRef
go back to reference Gleiss S, Kayser C (2013) Eccentricity dependent auditory enhancement of visual stimulus detection but not discrimination. Front Integr Neurosci 7:1–8 Gleiss S, Kayser C (2013) Eccentricity dependent auditory enhancement of visual stimulus detection but not discrimination. Front Integr Neurosci 7:1–8
go back to reference Harrington LK, Peck CK (1998) Spatial disparity affects visual–auditory interactions in human sensorimotor processing. Exp Brain Res 122:247–252PubMedCrossRef Harrington LK, Peck CK (1998) Spatial disparity affects visual–auditory interactions in human sensorimotor processing. Exp Brain Res 122:247–252PubMedCrossRef
go back to reference Hughes HC, Reuter-Lorenz PA, Nozawa G, Fendrich R (1994) Visual–auditory interactions in sensorimotor processing: saccades versus manual responses. J Exp Psychol Hum Percept Perform 20:131–153PubMedCrossRef Hughes HC, Reuter-Lorenz PA, Nozawa G, Fendrich R (1994) Visual–auditory interactions in sensorimotor processing: saccades versus manual responses. J Exp Psychol Hum Percept Perform 20:131–153PubMedCrossRef
go back to reference Kaplan E, Shapley RM (1986) The primate retina contains two types of ganglion cells, with high and low contrast sensitivity. Proc Natl Acad Sci USA 83:2755–2757PubMedCentralPubMedCrossRef Kaplan E, Shapley RM (1986) The primate retina contains two types of ganglion cells, with high and low contrast sensitivity. Proc Natl Acad Sci USA 83:2755–2757PubMedCentralPubMedCrossRef
go back to reference Kulikowski J, Tolhurst D (1973) Psychophysical evidence for sustained and transient detectors in human vision. J Physiol 232(1):149–162PubMedCentralPubMed Kulikowski J, Tolhurst D (1973) Psychophysical evidence for sustained and transient detectors in human vision. J Physiol 232(1):149–162PubMedCentralPubMed
go back to reference Legge GE (1978) Sustained and transient mechanisms in human vision: temporal and spatial properties. Vis Res 18:69–81PubMedCrossRef Legge GE (1978) Sustained and transient mechanisms in human vision: temporal and spatial properties. Vis Res 18:69–81PubMedCrossRef
go back to reference Lewkowicz D, Kraebel K (2004) The value of multisensory redundancy in the development of intersensory perception. In: Calvert GA, Spence C, Stein B (eds) Handbook of multisensory process. MIT Press, Cambridge, pp 655–678 Lewkowicz D, Kraebel K (2004) The value of multisensory redundancy in the development of intersensory perception. In: Calvert GA, Spence C, Stein B (eds) Handbook of multisensory process. MIT Press, Cambridge, pp 655–678
go back to reference Maunsell JH, Ghose GM, Assad JA et al (1999) Visual response latencies of magnocellular and parvocellular LGN neurons in macaque monkeys. Vis Neurosci 16:1–14PubMedCrossRef Maunsell JH, Ghose GM, Assad JA et al (1999) Visual response latencies of magnocellular and parvocellular LGN neurons in macaque monkeys. Vis Neurosci 16:1–14PubMedCrossRef
go back to reference Meredith MA, Stein BE (1996) Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75:1843–1857PubMed Meredith MA, Stein BE (1996) Spatial determinants of multisensory integration in cat superior colliculus neurons. J Neurophysiol 75:1843–1857PubMed
go back to reference Meredith M, Nemitz J, Stein B (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215–3229PubMed Meredith M, Nemitz J, Stein B (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215–3229PubMed
go back to reference Merigan WH (1989) Chromatic and achromatic vision of macaques: role of the P pathway. J Neurosci 9:776–783PubMed Merigan WH (1989) Chromatic and achromatic vision of macaques: role of the P pathway. J Neurosci 9:776–783PubMed
go back to reference Meyer GF, Wuerger SM (2001) Cross-modal integration of auditory and visual motion signals. NeuroReport 12:2557–2560PubMedCrossRef Meyer GF, Wuerger SM (2001) Cross-modal integration of auditory and visual motion signals. NeuroReport 12:2557–2560PubMedCrossRef
go back to reference Miller J (1982) Divided attention: evidence for coactivation with redundant signals. Cogn Psychol 14:247–279PubMedCrossRef Miller J (1982) Divided attention: evidence for coactivation with redundant signals. Cogn Psychol 14:247–279PubMedCrossRef
go back to reference Miller J (1986) Timecourse of coactivation in bimodal divided attention. Percept Psychophys 40:331–343PubMedCrossRef Miller J (1986) Timecourse of coactivation in bimodal divided attention. Percept Psychophys 40:331–343PubMedCrossRef
go back to reference Molholm S, Ritter W, Murray MM et al (2002) Multisensory auditory–visual interactions during early sensory processing in humans: a high-density electrical mapping study. Cogn Brain Res 14:115–128CrossRef Molholm S, Ritter W, Murray MM et al (2002) Multisensory auditory–visual interactions during early sensory processing in humans: a high-density electrical mapping study. Cogn Brain Res 14:115–128CrossRef
go back to reference Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15:963-974PubMedCrossRef Murray MM, Molholm S, Michel CM, Heslenfeld DJ, Ritter W, Javitt DC, Schroeder CE, Foxe JJ (2005) Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb Cortex 15:963-974PubMedCrossRef
go back to reference Odgaard EC, Arieh Y, Marks LE (2003) Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percept Psychophys 65:123–132PubMedCrossRef Odgaard EC, Arieh Y, Marks LE (2003) Cross-modal enhancement of perceived brightness: sensory interaction versus response bias. Percept Psychophys 65:123–132PubMedCrossRef
go back to reference Perrott DR, Saberi K, Brown K, Strybel TZ (1990) Auditory psychomotor coordination and visual search performance. Percept Psychophys 48:214–226PubMedCrossRef Perrott DR, Saberi K, Brown K, Strybel TZ (1990) Auditory psychomotor coordination and visual search performance. Percept Psychophys 48:214–226PubMedCrossRef
go back to reference Rockland KS, Ojima H (2003) Multisensory convergence in calcarine visual areas in macaque monkey. Int J Psychophysiol 50:19–26PubMedCrossRef Rockland KS, Ojima H (2003) Multisensory convergence in calcarine visual areas in macaque monkey. Int J Psychophysiol 50:19–26PubMedCrossRef
go back to reference Schiller PH, Malpeli JG, Schein SJ (1979) Composition of geniculostriate input to superior colliculus of the rhesus monkey. J Neurophysiol 42:1124–1133PubMed Schiller PH, Malpeli JG, Schein SJ (1979) Composition of geniculostriate input to superior colliculus of the rhesus monkey. J Neurophysiol 42:1124–1133PubMed
go back to reference Schroeder CE, Mehta AD, Givre SJ (1998) A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cereb Cortex 8:575–592PubMedCrossRef Schroeder CE, Mehta AD, Givre SJ (1998) A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cereb Cortex 8:575–592PubMedCrossRef
go back to reference Thurlow WR, Jack CE (1973) Some determinants of localization–adaptation effects for successive auditory stimuli. J Acoust Soc Am 53:1573–1577PubMedCrossRef Thurlow WR, Jack CE (1973) Some determinants of localization–adaptation effects for successive auditory stimuli. J Acoust Soc Am 53:1573–1577PubMedCrossRef
go back to reference Tootell RB, Silverman MS, Hamilton SL et al (1988) Functional anatomy of macaque striate cortex. V. Spatial frequency. J Neurosci 8:1610–1624PubMed Tootell RB, Silverman MS, Hamilton SL et al (1988) Functional anatomy of macaque striate cortex. V. Spatial frequency. J Neurosci 8:1610–1624PubMed
go back to reference Vroomen J, de Gelder B (2000) Sound enhances visual perception: cross-modal effects of auditory organization on vision. J Exp Psychol Hum Percept Perform 26:1583–1590PubMedCrossRef Vroomen J, de Gelder B (2000) Sound enhances visual perception: cross-modal effects of auditory organization on vision. J Exp Psychol Hum Percept Perform 26:1583–1590PubMedCrossRef
go back to reference Walker JT, Scott KJ (1981) Auditory–visual conflicts in the perceived duration of lights, tones and gaps. J Exp Psychol Hum Percept Perform 7:1327–1339PubMedCrossRef Walker JT, Scott KJ (1981) Auditory–visual conflicts in the perceived duration of lights, tones and gaps. J Exp Psychol Hum Percept Perform 7:1327–1339PubMedCrossRef
go back to reference Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246–1266PubMed Wallace MT, Wilkinson LK, Stein BE (1996) Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76:1246–1266PubMed
go back to reference Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80:1006–1010PubMed Wallace MT, Meredith MA, Stein BE (1998) Multisensory integration in the superior colliculus of the alert cat. J Neurophysiol 80:1006–1010PubMed
go back to reference Wilkinson LK, Meredith MA, Stein BE (1996) The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp Brain Res. doi:10.1007/BF00227172 PubMed Wilkinson LK, Meredith MA, Stein BE (1996) The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp Brain Res. doi:10.​1007/​BF00227172 PubMed
go back to reference Wise L, Irvine D (1983) Auditory response properties of neurons in deep layers of cat superior colliculus. J Neurophysiol 49:674–685PubMed Wise L, Irvine D (1983) Auditory response properties of neurons in deep layers of cat superior colliculus. J Neurophysiol 49:674–685PubMed
Metadata
Title
On the ‘visual’ in ‘audio-visual integration’: a hypothesis concerning visual pathways
Authors
Philip Jaekl
Alexis Pérez-Bellido
Salvador Soto-Faraco
Publication date
01-06-2014
Publisher
Springer Berlin Heidelberg
Published in
Experimental Brain Research / Issue 6/2014
Print ISSN: 0014-4819
Electronic ISSN: 1432-1106
DOI
https://doi.org/10.1007/s00221-014-3927-8

Other articles of this Issue 6/2014

Experimental Brain Research 6/2014 Go to the issue