Top

Published in:

01-11-2019 | Original Article

An investigation of the neural association between auditory imagery and perception of complex sounds

Authors: Jin Gu, Hairuo Zhang, Baolin Liu, Xianglin Li, Peiyuan Wang, Bin Wang

Published in: Brain Structure and Function | Issue 8/2019

Abstract

Neuroimaging studies have demonstrated that mental imagery and perception share similar neural substrates, however, there are still ambiguities according to different auditory imagery content. In addition, there is still a lack of information regarding the underlying neural correlation between the two modalities. In the present study, we adopted functional magnetic resonance imaging to explore the neural representation during imagery and perception of actual sounds in our surroundings. Univariate analysis was used to assess the differences between the modalities of average activation intensity, and stronger imagery activation was found in sensorimotor regions but weaker activation in auditory association cortices. Additionally, multi-voxel pattern analysis with a support vector machine classifier was implemented to decode environmental sounds within- or cross-modality. Significant above-chance accuracies were found in all overlapping regions in the classification of within-modality, while successful cross-modality classification only was found in sensorimotor regions. Both univariate and multivariate analyses found distinct representation between auditory imagery and perception in the overlapping regions, including superior temporal gyrus and inferior frontal sulcus as well as the precentral cortex and pre-supplementary motor area. Our results confirm the overlapping activation regions between auditory imagery and perception reported by previous studies and suggest that activation regions showed dissociable representation pattern in imagery and perception of sound categories.

Addis DR, Pan L, Vu MA, Laiser N, Schacter DL (2009) Constructive episodic simulation of the future and the past: distinct subsystems of a core brain network mediate imagining and remembering. Neuropsychologia 47(11):2222–2238. https://doi.org/10.1016/j.neuropsychologia.2008.10.026 CrossRefPubMed

Albers AM, Kok P, Toni I, Dijkerman HC, de Lange FP (2013) Shared representations for working memory and mental imagery in early visual cortex. Curr Biol 23(15):1427–1431. https://doi.org/10.1016/j.cub.2013.05.065 CrossRefPubMed

Amaro E, Barker GJ (2006) Study design in fMRI: basic principles. Brain Cogn 60(3):220–232CrossRefPubMed

Amunts K, Morosan P, Hilbig H, Zilles K (2012) Auditory system. Hum Nerv Syst. https://doi.org/10.1016/b978-0-12-374236-0.10036-7 CrossRef

Ariani G, Oosterhof NN, Lingnau A (2018) Time-resolved decoding of planned delayed and immediate prehension movements. Cortex 99:330–345. https://doi.org/10.1016/j.cortex.2017.12.007 CrossRefPubMed

Arnott SR, Grady CL, Hevenor SJ, Graham S, Alain C (2005) The functional organization of auditory working memory as revealed by fMRI. J Cogn Neurosci 17(5):819–831. https://doi.org/10.1162/0898929053747612 CrossRefPubMed

Boccia M, Sulpizio V, Palermo L, Piccardi L, Guariglia C, Galati G (2017) I can see where you would be: patterns of fMRI activity reveal imagined landmarks. NeuroImage 144:174–182CrossRefPubMed

Bunzeck N, Wuestenberg T, Lutz K, Heinze HJ, Jancke L (2005) Scanning silence: mental imagery of complex sounds. NeuroImage 26(4):1119–1127. https://doi.org/10.1016/j.neuroimage.2005.03.013 CrossRefPubMed

Cichy RM, Heinzle J, Haynes JD (2012) Imagery and perception share cortical representations of content and location. Cereb Cortex 22:372–380CrossRefPubMed

Copolov DL, Seal ML, Maruff P, Ulusoy R, Wong MT, Tochon-Danguy HJ, Egan GF (2003) Cortical activation associated with the experience of auditory hallucinations and perception of human speech in schizophrenia: a PET correlation study. Psychiatry Res 122(3):139–152CrossRefPubMed

Davis T, LaRocque KF, Mumford JA, Norman KA, Wagner AD, Poldrack RA (2014) What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis. NeuroImage 97:271–283CrossRefPubMedPubMedCentral

de Borst AW, de Gelder B (2018) Mental imagery follows similar cortical reorganization as perception: intra-modal and cross-modal plasticity in congenitally blind. Cereb Cortex. https://doi.org/10.1093/cercor/bhy151 CrossRef

de Borst AW, Valente G, Jaaskelainen IP, Tikka P (2016) Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals. NeuroImage 129:428–438. https://doi.org/10.1016/j.neuroimage.2016.01.043 CrossRefPubMed

Dierks T, Linden DE, Jandl M, Formisano E, Goebel R, Lanfermann H, Singer W (1999) Activation of Heschl’s gyrus during auditory hallucinations. Neuron 22(3):615–621CrossRefPubMed

Emmerling TC, Zimmermann J, Sorger B, Frost MA, Goebel R (2016) Decoding the direction of imagined visual motion using 7T ultra-high field fMRI. NeuroImage 125:61–73. https://doi.org/10.1016/j.neuroimage.2015.10.022 CrossRefPubMedPubMedCentral

Engel LR, Frum C, Puce A, Walker NA, Lewis JW (2009) Different categories of living and non-living sound-sources activate distinct cortical networks. NeuroImage 47:1778–1791CrossRefPubMedPubMedCentral

Ganis G, Thompson WL, Kosslyn SM (2004) Brain areas underlying visual mental imagery and visual perception: an fMRI study. Brain Res Cogn Brain Res 20(2):226–241. https://doi.org/10.1016/j.cogbrainres.2004.02.012 CrossRefPubMed

Gilbert SJ, Fung H (2018) Decoding intentions of self and others from fMRI activity patterns. NeuroImage 172:278–290. https://doi.org/10.1016/j.neuroimage.2017.12.090 CrossRefPubMed

Halpern AR, Zatorre RJ (1999) When that tune runs through your head: a pet investigation of auditory imagery for familiar melodies [J]. Cereb Cortex 9(7):697–704CrossRefPubMed

Halpern AR, Zatorre RJ, Bouffard M, Johnson JA (2004) Behavioral and neural correlates of perceived and imagined musical timbre. Neuropsychologia 42(9):1281–1292. https://doi.org/10.1016/j.neuropsychologia.2003.12.017 CrossRefPubMed

Hampshire A, Chamberlain SR, Monti MM, Duncan J, Owen AM (2010) The role of the right inferior frontal gyrus: inhibition and attentional control. NeuroImage 20:1313–1319CrossRef

Herholz SC, Halpern AR, Zatorre RJ (2012) Neuronal correlates of perception, imagery, and memory for familiar tunes. J Cogn Neurosci 24(6):1382–1397. https://doi.org/10.1162/jocn_a_00216 CrossRefPubMed

Hickok G, Poeppel D (2007) The cortical organization of speech processing. Nat Rev Neurosci 8(5):393CrossRefPubMed

Huang S, Seidman LJ, Rossi S, Ahveninen J (2013) Distinct cortical networks activated by auditory attention and working memory load. NeuroImage 83:1098–1108. https://doi.org/10.1016/j.neuroimage.2013.07.074 CrossRefPubMed

Hubbard TL (2010) Auditory imagery: empirical findings. Psychol Bull 136(2):302–329. https://doi.org/10.1037/a0018436 CrossRefPubMed

Huijbers W, Pennartz CM, Rubin DC, Daselaar SM (2011) Imagery and retrieval of auditory and visual information: neural correlates of successful and unsuccessful performance. Neuropsychologia 49(7):1730–1740. https://doi.org/10.1016/j.neuropsychologia.2011.02.051 CrossRefPubMed

Jäncke Lutz, Shah NJ (2004) ‘Hearing’ syllables by ‘seeing’ visual stimuli. Eur J Neurosci 19(9):2603–2608CrossRefPubMed

Jardri R, Pouchet A, Pins D, Thomas P (2011) Cortical activations during auditory verbal hallucinations in schizophrenia: a coordinate-based meta-analysis. Am J Psychiatry 168(1):73–81. https://doi.org/10.1176/appi.ajp.2010.09101522 CrossRefPubMed

Koelsch S, Schulze K, Sammler D, Fritz T, Muller K, Gruber O (2009) Functional architecture of verbal and tonal working memory: an FMRI study. Hum Brain Mapp 30(3):859–873. https://doi.org/10.1002/hbm.20550 CrossRefPubMed

Kompus K, Westerhausen R, Hugdahl K (2011) The “paradoxical” engagement of the primary auditory cortex in patients with auditory verbal hallucinations: a meta-analysis of functional neuroimaging studies. Neuropsychologia 49(12):3361–3369. https://doi.org/10.1016/j.neuropsychologia.2011.08.010 CrossRefPubMed

Kraemer DJ, Macrae CN, Green AE, Kelley WM (2005) Musical imagery: sound of silence activates auditory cortex. Nature 434(7030):158CrossRefPubMed

Kriegeskorte N, Simmons WK, Bellgowan PS, Baker CI (2009) Circular analysis in systems neuroscience: the dangers of double dipping. Nat Neurosci 12(5):535–540. https://doi.org/10.1038/nn.2303 CrossRefPubMedPubMedCentral

Kumar S, Sedley W, Barnes GR, Teki S, Friston KJ, Griffiths TD (2014) A brain basis for musical hallucinations. Cortex 52:86–97. https://doi.org/10.1016/j.cortex.2013.12.002 CrossRefPubMedPubMedCentral

Le Bihan D, Turner R, Zeffiro TA, Cuenod CA, Jezzard P, Bonnerot V (1993) Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study. Proc Natl Acad Sci USA 90(24):11802–11805CrossRefPubMed

Lee SH, Kravitz DJ, Baker CI (2012) Disentangling visual imagery and perception of real-world objects. NeuroImage 59(4):4064–4073CrossRefPubMed

Leung AW, Alain C (2011) Working memory load modulates the auditory “What” and “Where” neural networks. NeuroImage 55(3):1260–1269. https://doi.org/10.1016/j.neuroimage.2010.12.055 CrossRefPubMed

Lewis JW, Brefczynski JA, Phinney RE, Janik JJ, DeYoe EA (2005) Distinct cortical pathways for processing tool versus animal sounds. J Neurosci 25:5148–5158CrossRefPubMedPubMedCentral

Lima CF, Lavan N, Evans S (2015) Feel the noise: relating individual differences in auditory imagery to the structure and function of sensorimotor systems. Cereb Cortex 25:4638–4650CrossRefPubMedPubMedCentral

Lima CF, Krishnan S, Scott SK (2016) Roles of supplementary motor areas in auditory processing and auditory imagery. Trends Neurosci 39(8):527–542. https://doi.org/10.1016/j.tins.2016.06.003 CrossRefPubMedPubMedCentral

Linke AC, Cusack R (2015) Flexible information coding in human auditory cortex during perception, imagery, and STM of complex sounds. J Cogn Neurosci 27(7):1322–1333. https://doi.org/10.1162/jocn_a_00780 CrossRefPubMed

Maeder PP, Meuli RA, Adriani M, Bellmann A, Fornari E, Thiran JP, Pittet A, Clarke S (2001) Distinct pathways involved in sound recognition and localization: a human fMRI study. NeuroImage 14:802–816CrossRefPubMed

Mahmoudi A, Takerkart S, Regragui F, Boussaoud D, Brovelli A (2012) Multivoxel pattern analysis for FMRI data: a review. Comput Math Methods Med 2012:1–14CrossRef

McGettigan C, Walsh E, Jessop R, Agnew ZK, Sauter DA, Warren JE, Scott SK (2015) Individual differences in laughter perception reveal roles for mentalizing and sensorimotor systems in the evaluation of emotional authenticity. Cereb Cortex 25:246–257CrossRefPubMed

Meteyard L, Cuadrado SR, Bahrami B, Vigliocco G (2012) Coming of age: a review of embodiment and the neuroscience of semantics. Cortex 48:788–804CrossRefPubMed

Norman KA, Polyn SM, Detre GJ, Haxby JV (2006) Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn Sci 10:424–430CrossRefPubMed

Oh J, Kwon JH, Yang PS, Jeong J (2013) Auditory imagery modulates frequency-specific areas in the human auditory cortex. J Cogn Neurosci 25(2):175–187. https://doi.org/10.1162/jocn_a_00280 CrossRefPubMed

Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9(1):97–113CrossRef

Raij TT, Riekki TJ (2012) Poor supplementary motor area activation differentiates auditory verbal hallucination from imagining the hallucination. Neuroimage Clin 1(1):75–80CrossRefPubMedPubMedCentral

Rauschecker JP, Scott SK (2009) Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nat Neurosci 12(6):718–724. https://doi.org/10.1038/nn.2331 CrossRefPubMedPubMedCentral

Shergill SS, Bullmore ET, Brammer MJ (2001) A functional study of auditory verbal imagery. Psychol Med 31(2):241. https://doi.org/10.1017/S003329170100335X CrossRefPubMed

Stelzer J, Chen Y, Turner R (2013) Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): random permutations and cluster size control. NeuroImage 65:69–82. https://doi.org/10.1016/j.neuroimage.2012.09.063 CrossRefPubMed

Stuart GP, Jones DM (1996) From auditory image to auditory percept: facilitation through common processes? Mem Cognit 24:296–304CrossRefPubMed

Tian B, Reser D, Amy D (2001) Functional specialization in rhesus monkey auditory cortex. Science 292(5515):290–293CrossRefPubMed

Tian X, Zarate JM, Poeppel D (2016) Mental imagery of speech implicates two mechanisms of perceptual reactivation. Cortex 77:1–12. https://doi.org/10.1016/j.cortex.2016.01.002 CrossRefPubMedPubMedCentral

Tong Y, Chen Q, Nichols TE, Rasetti R, Callicott JH, Berman KF, Weinberger DR, Mattay VS (2016) Seeking optimal region-of-interest (ROI) single-value summary measures for fmri studies in imaging genetics. PLoS One 11:e151391

Treder MS, Purwins H, Miklody D, Sturm I, Blankertz B (2014) Decoding auditory attention to instruments in polyphonic music using single-trial EEG classification. J Neural Eng 11(2):0226009CrossRef

van de Ven VG, Formisano E, Roder CH, Prvulovic D, Bittner RA, Dietz MG, Hubl D, Dierks T, Federspiel A, Esposito F, Di Salle F, Jansma B, Goebel R, Linden DE (2005) The spatiotemporal pattern of auditory cortical responses during verbal hallucinations. NeuroImage 27(3):644–655. https://doi.org/10.1016/j.neuroimage.2005.04.041 CrossRefPubMed

Vetter P, Smith FW, Muckli L (2014) Decoding sound and imagery content in early visual cortex. Curr Biol 24(11):1256–1262. https://doi.org/10.1016/j.cub.2014.04.020 CrossRefPubMedPubMedCentral

Wang X, Gu J, Xu J, Li X, Geng J, Wang B, Liu B (2018) Decoding natural scenes based on sounds of objects within scenes using multivariate pattern analysis. Neurosci Res. https://doi.org/10.1016/j.neures.2018.11.009 CrossRefPubMed

Wheeler ME, Petersen SE, Buckner RL (2000) Memory’s echo: Vivid remembering reactivates sensory-specific cortex. Proc Natl Acad Sci India Sect A (Phys Sci) 97(20):11125–11129CrossRef

Willander J, Baraldi S (2010) Development of a new Clarity of Auditory Imagery Scale. Behav Res Methods 42(3):785–790CrossRefPubMed

Wong D, Pisoni DB, Learn J (2002) PET imaging of differential cortical activation by monaural speech and nonspeech stimuli. Hear Res 166(1):9–23CrossRefPubMedPubMedCentral

Wu J, Mai X, Chan CC, Zheng Y, Luo Y (2006) Event-related potentials during mental imagery of animal sounds. Psychophysiology 43:592–597CrossRefPubMed

Wu J, Yu Z, Mai X, Wei J, Luo Y (2011) Pitch and loudness information encoded in auditory imagery as revealed by event-related potentials. Psychophysiology 48(3):415–419. https://doi.org/10.1111/j.1469-8986.2010.01070.x CrossRefPubMed

Yoo SS, Lee CU, Choi BG (2001) Human brain mapping of auditory imagery: event-related functional MRI study. NeuroReport 12(14):3045–3049CrossRefPubMed

Yoo SS, Freeman DK, McCarthy JJ, Jolesz FA (2003) Neural substrates of tactile imagery: a functional MRI study. NeuroReport 14(4):581–585CrossRefPubMed

Zafar R, Malik AS, Kamel N, Dass SC, Abdullah JM, Reza F, Abdul KA (2015) Decoding of visual information from human brain activity: a review of fMRI and EEG studies. J Integra Neurosci 14:155–168CrossRef

Zald DH, Pardo JV (2002) The neural correlates of aversive auditory stimulation. NeuroImage 16:746–753CrossRefPubMed

Zatorre RJ, Evans AC, Meyer E (1994) Neural mechanisms underlying melodic perception and memory for pitch. J Neurosci Methods 14:1908–1919CrossRef

Zeidman P, Mullally SL, Maguire EA (2015) Constructing, perceiving, and maintaining scenes: hippocampal activity and connectivity. Cereb Cortex 25(10):3836–3855. https://doi.org/10.1093/cercor/bhu266 CrossRefPubMed

Zmigrod L, Garrison JR, Carr J, Simons JS (2016) The neural mechanisms of hallucinations: a quantitative meta-analysis of neuroimaging studies. Neurosci Biobehav Rev 69:113–123. https://doi.org/10.1016/j.neubiorev.2016.05.037 CrossRefPubMed

Title: An investigation of the neural association between auditory imagery and perception of complex sounds
Authors: Jin Gu
Hairuo Zhang
Baolin Liu
Xianglin Li
Peiyuan Wang
Bin Wang
Publication date: 01-11-2019
Publisher: Springer Berlin Heidelberg
Published in: Brain Structure and Function / Issue 8/2019
Print ISSN: 1863-2653
Electronic ISSN: 1863-2661
DOI: https://doi.org/10.1007/s00429-019-01948-z

At a glance: The STEP trials

Springer Medicine

An investigation of the neural association between auditory imagery and perception of complex sounds

Abstract

At a glance: The STEP trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 8/2019

Receptor density pattern confirms and enhances the anatomic-functional features of the macaque superior parietal lobule areas

Functional neuroanatomy of the human eye movement network: a review and atlas

Loss of serum response factor in mature neurons in the dentate gyrus alters the morphology of dendritic spines and hippocampus-dependent behavioral tasks

Mapping the landscape of human dopamine D2/3 receptors with [11C]raclopride

The time-course of thermoregulatory responses during treadmill running is associated with running duration-dependent hypothalamic neuronal activation in rats

Reducing negative affect with anodal transcranial direct current stimulation increases memory performance in young—but not in elderly—individuals