Top

Published in:

01-03-2018 | Neurology and Preclinical Neurological Studies - Review Article

Parallel basal ganglia circuits for decision making

Authors: Okihide Hikosaka, Ali Ghazizadeh, Whitney Griggs, Hidetoshi Amita

Published in: Journal of Neural Transmission | Issue 3/2018

Abstract

The basal ganglia control body movements, mainly, based on their values. Critical for this mechanism is dopamine neurons, which sends unpredicted value signals, mainly, to the striatum. This mechanism enables animals to change their behaviors flexibly, eventually choosing a valuable behavior. However, this may not be the best behavior, because the flexible choice is focused on recent, and, therefore, limited, experiences (i.e., short-term memories). Our old and recent studies suggest that the basal ganglia contain separate circuits that process value signals in a completely different manner. They are insensitive to recent changes in value, yet gradually accumulate the value of each behavior (i.e., movement or object choice). These stable circuits eventually encode values of many behaviors and then retain the value signals for a long time (i.e., long-term memories). They are innervated by a separate group of dopamine neurons that retain value signals, even when no reward is predicted. Importantly, the stable circuits can control motor behaviors (e.g., hand or eye) quickly and precisely, which allows animals to automatically acquire valuable outcomes based on historical life experiences. These behaviors would be called ‘skills’, which are crucial for survival. The stable circuits are localized in the posterior part of the basal ganglia, separately from the flexible circuits located in the anterior part. To summarize, the flexible and stable circuits in the basal ganglia, working together but independently, enable animals (and humans) to reach valuable goals in various contexts.

Aarts H, Dijksterhuis A (2000) Habits as knowledge structures: automaticity in goal-directed behavior. J Pers Soc Psychol 78:53–63CrossRefPubMed

Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357–381CrossRefPubMed

Ammons RB, Farr RG, Bloch E, Neumann E, Dey M, Marion R, Ammons CH (1958) Long-term retention of perceptual-motor skills. J Exp Psychol 55:318–328CrossRefPubMed

Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacol 37:407–419CrossRef

Balleine BW, O'Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacol 35:48–69CrossRef

Bergman H, Feingold A, Nini A, Raz A, Slovin H, Abeles M, Vaadia E (1998) Physiological aspects of information processing in the basal ganglia of normal and parkinsonian primates. Trends Neurosci 21:32–38CrossRefPubMed

Bhatia KP, Marsden CD (1994) The behavioural and motor consequences of focal lesions of the basal ganglia in man. Brain 117:859–876CrossRefPubMed

Bichot NP, Schall JD (1999) Effects of similarity and history on neural mechanisms of visual selection. Nat Neurosci 2:549–554CrossRefPubMed

Bromberg-Martin ES, Hikosaka O (2009) Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron 63:119–126CrossRefPubMedPubMedCentral

Brown LL, Schneider JS, Lidsky TI (1997) Sensory and cognitive functions of the basal ganglia. Curr Opin Neurobiol 7:157–163CrossRefPubMed

Cheramy A, Leviel V, Glowinski J (1981) Dendritic release of dopamine in the substantia nigra. Nature 289:537–542CrossRefPubMed

Cowan N (2001) The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav Brain Sci 24:87–114 (discussion 114–185) CrossRefPubMed

Ericsson KA, Lehmann AC (1996) Expert and exceptional performance: evidence of maximal adaptation to task constraints. Annu Rev Psychol 47:273–305CrossRefPubMed

Flaherty AW, Graybiel AM (1991) Corticostriatal transformations in the primate somatosensory system. Projections from physiologically mapped body-part representations. J Neurophysiol 66:1249–1263CrossRefPubMed

François C, Yelnik J, Percheron G, Fénelon G (1994) Topographic distribution of the axonal endings from the sensorimotor and associative striatum in the macaque pallidum and substantia nigra. Exp Brain Res 102:305–318CrossRefPubMed

Ghazizadeh A, Griggs W, Hikosaka O (2016a) Object-finding skill created by repeated reward experience. J Vis 16:17CrossRefPubMedPubMedCentral

Ghazizadeh A, Griggs W, Hikosaka O (2016b) Ecological origins of object salience: reward, uncertainty, aversiveness, and novelty. Front Neurosci 10:378CrossRefPubMedPubMedCentral

Grupe DW, Nitschke JB (2013) Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective. Nat Rev Neurosci 14:488–501CrossRefPubMedPubMedCentral

Haber SN, Kim KS, Mailly P, Calzavara R (2006) Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci 26:8368–8376CrossRefPubMed

Hikosaka O, Wurtz RH (1983) Visual and oculomotor functions of monkey substantia nigra pars reticulata. III. Memory-contingent visual and saccade responses. J Neurophysiol 49:1268–1284CrossRefPubMed

Hikosaka O, Sakamoto M, Usui S (1989a) Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. J Neurophysiol 61:780–798CrossRefPubMed

Hikosaka O, Sakamoto M, Usui S (1989b) Functional properties of monkey caudate neurons. II. Visual and auditory responses. J Neurophysiol 61:799–813CrossRefPubMed

Hikosaka O, Sakamoto M, Usui S (1989c) Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61:814–832CrossRefPubMed

Hikosaka O, Rand MK, Miyachi S, Miyashita K (1995) Learning of sequential movements in the monkey: process of learning and retention of memory. J Neurophysiol 74:1652–1661CrossRefPubMed

Hikosaka O, Nakahara H, Rand MK, Sakai K, Lu X, Nakamura K, Miyachi S, Doya K (1999) Parallel neural networks for learning sequential procedures. Trends Neurosci 22:464–471CrossRefPubMed

Hikosaka O, Takikawa Y, Kawagoe R (2000) Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev 80:953–978CrossRefPubMed

Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002a) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217–222CrossRefPubMed

Hikosaka O, Rand MK, Nakamura K, Miyachi S, Kitaguchi K, Sakai K, Lu X, Shimo Y (2002b) Long-term retention of motor skill in macaque monkeys and humans. Exp Brain Res 147:494–504CrossRefPubMed

Hikosaka O, Nakamura K, Nakahara H (2006) Basal ganglia orient eyes to reward. J Neurophysiol 95:567–584CrossRefPubMed

Hikosaka O, Yamamoto S, Yasuda M, Kim HF (2013) Why skill matters. Trends Cogn Sci 17:434–441CrossRefPubMedPubMedCentral

Hikosaka O, Kim HF, Yasuda M, Yamamoto S (2014) Basal Ganglia circuits for reward value-guided behavior. Annu Rev Neurosci 37:289–306CrossRefPubMedPubMedCentral

Isoda M, Hikosaka O (2007) Switching from automatic to controlled action by monkey medial frontal cortex. Nat Neurosci 10:240–248CrossRefPubMed

Isoda M, Hikosaka O (2008) Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement. J Neurosci 28:7209–7218CrossRefPubMedPubMedCentral

Jankowski J, Scheef L, Huppe C, Boecker H (2009) Distinct striatal regions for planning and executing novel and automated movement sequences. NeuroImage 44:1369–1379CrossRefPubMed

Kaneda K, Nambu A, Tokuno H, Takada M (2002) Differential processing patterns of motor information via striatopallidal and striatonigral projections. J Neurophysiol 88:1420–1432CrossRefPubMed

Kato M, Miyashita N, Hikosaka O, Matsumura M, Usui S, Kori A (1995) Eye movements in monkeys with local dopamine depletion in the caudate nucleus. I. Deficits in spontaneous saccades. J Neurosci 15:912–927PubMed

Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411–416CrossRefPubMed

Kawagoe R, Takikawa Y, Hikosaka O (2004) Reward-predicting activity of dopamine and caudate neurons—a possible mechanism of motivational control of saccadic eye movement. J Neurophysiol 91:1013–1024CrossRefPubMed

Kim HF, Hikosaka O (2013) Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values. Neuron 79:1001–1010CrossRefPubMedPubMedCentral

Kim HF, Hikosaka O (2015) Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards. Brain 138:1776–1800CrossRefPubMedPubMedCentral

Kim HF, Ghazizadeh A, Hikosaka O (2014) Separate groups of dopamine neurons innervate caudate head and tail encoding flexible and stable value memories. Front Neuroanat 8:120CrossRefPubMedPubMedCentral

Kim HF, Ghazizadeh A, Hikosaka O (2015) Dopamine neurons encoding long-term memory of object value for habitual behavior. Cell 163:1165–1175CrossRefPubMedPubMedCentral

Kori A, Miyashita N, Kato M, Hikosaka O, Usui S, Matsumura M (1995) Eye movements in monkeys with local dopamine depletion in the caudate nucleus. II. Deficits in voluntary saccades. J Neurosci 15:928–941PubMed

Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural correlate of response bias in monkey caudate nucleus. Nature 418:413–417CrossRefPubMed

Lehericy S, Benali H, Van de Moortele PF, Pelegrini-Issac M, Waechter T, Ugurbil K, Doyon J (2005) Distinct basal ganglia territories are engaged in early and advanced motor sequence learning. Proc Natl Acad Sci USA 102:12566–12571CrossRefPubMedPubMedCentral

Lerner TN, Shilyansky C, Davidson TJ, Evans KE, Beier KT, Zalocusky KA, Crow AK, Malenka RC, Luo L, Tomer R, Deisseroth K (2015) Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits. Cell 162:635–647CrossRefPubMedPubMedCentral

Lu X, Hikosaka O, Miyachi S (1998) Role of monkey cerebellar nuclei in skill for sequential movement. J Neurophysiol 79:2245–2254CrossRefPubMed

Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459:837–841CrossRefPubMedPubMedCentral

Menegas W, Bergan JF, Ogawa SK, Isogai Y, Umadevi Venkataraju K, Osten P, Uchida N, Watabe-Uchida M (2015) Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass. Elife 4:e10032CrossRefPubMedPubMedCentral

Menegas W, Babayan BM, Uchida N, Watabe-Uchida M (2017) Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice. elife 6:e21886CrossRefPubMedPubMedCentral

Middleton FA, Strick PL (1996) The temporal lobe is a target of output from the basal ganglia. Proc Natl Acad Sci USA 93:8683–8687CrossRefPubMedPubMedCentral

Miyachi S, Hikosaka O, Miyashita K, Karádi Z, Rand MK (1997) Differential roles of monkey striatum in learning of sequential hand movement. Exp Brain Res 115:1–5CrossRefPubMed

Miyachi S, Hikosaka O, Lu X (2002) Differential activation of monkey striatal neurons in the early and late stages of procedural learning. Exp Brain Res 146:122–126CrossRefPubMed

Miyashita N, Hikosaka O, Kato M (1995) Visual hemineglect induced by unilateral striatal dopamine deficiency in monkeys. NeuroReport 6:1257–1260CrossRefPubMed

Miyashita K, Rand MK, Miyachi S, Hikosaka O (1996) Anticipatory saccades in sequential procedural learning in monkeys. J Neurophysiol 76:1361–1366CrossRefPubMed

Myers RE (1976) Comparative neurology of vocalization and speech: proof of a dichotomy. Ann N Y Acad Sci 280:745–760CrossRefPubMed

Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280CrossRefPubMed

Nakamura K, Hikosaka O (2006) Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J Neurosci 26:5360–5369CrossRefPubMed

Nakamura K, Sakai K, Hikosaka O (1998) Neuronal activity in medial frontal cortex during learning of sequential procedures. J Neurophysiol 80:2671–2687CrossRefPubMed

Nakamura K, Sakai K, Hikosaka O (1999) Effects of local inactivation of monkey medial frontal cortex in learning of sequential procedures. J Neurophysiol 82:1063–1068CrossRefPubMed

Nicola SM, Surmeier DJ, Malenka RC (2000) Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu Rev Neurosci 23:185–215CrossRefPubMed

Parent A (1990) Extrinsic connections of the basal ganglia. Trends Neurosci 13:254–258CrossRefPubMed

Rand MK, Hikosaka O, Miyachi S, Lu X, Miyashita K (1998) Characteristics of a long-term procedural skill in the monkey. Exp Brain Res 118:293–297CrossRefPubMed

Rand MK, Hikosaka O, Miyachi S, Lu X, Nakamura K, Kitaguchi K, Shimo Y (2000) Characteristics of sequential movements during early learning period in monkeys. Exp Brain Res 131:293–304CrossRefPubMed

Redgrave P, Rodriguez M, Smith Y, Rodriguez-Oroz MC, Lehericy S, Bergman H, Agid Y, DeLong MR, Obeso JA (2010) Goal-directed and habitual control in the basal ganglia: implications for Parkinson’s disease. Nat Rev Neurosci 11:760–772CrossRefPubMedPubMedCentral

Reynolds JN, Wickens JR (2002) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507–521CrossRefPubMed

Richfield EK, Young AB, Penney JB (1987) Comparative distribution of dopamine D-1 and D-2 receptors in the basal ganglia of turtles, pigeons, rats, cats, and monkeys. J Comp Neurol 262:446–463CrossRefPubMed

Saint-Cyr JA, Ungerleider LG, Desimone R (1990) Organization of visual cortical inputs to the striatum and subsequent outputs to the pallido-nigral complex in the monkey. J Comp Neurol 298:129–156CrossRefPubMed

Sakai K, Hikosaka O, Miyauchi S, Takino R, Sasaki Y, Pütz B (1998) Transition of brain activation from frontal to parietal areas in visuo-motor sequence learning. J Neurosci 18:1827–1840PubMed

Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27CrossRefPubMed

Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12:4595–4610PubMed

Selemon LD, Goldman-Rakic PS (1985) Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey. J Neurosci 5:776–794PubMed

Sidibe M, Pare JF, Smith Y (2002) Nigral and pallidal inputs to functionally segregated thalamostriatal neurons in the centromedian/parafascicular intralaminar nuclear complex in monkey. J Comp Neurol 447:286–299CrossRefPubMed

Skinner BF (1963) Operant behavior. Am Psychol 18:503–515

Smith Y, Parent A (1986) Differential connections of caudate nucleus and putamen in the squirrel monkey (Saimiri sciureus). Neuroscience 18:347–371CrossRefPubMed

Strasburger H, Rentschler I, Juttner M (2011) Peripheral vision and pattern recognition: a review. J Vis 11:13CrossRefPubMed

Takada M, Tokuno H, Nambu A, Inase M (1998) Corticostriatal projections from the somatic motor areas of the frontal cortex in the macaque monkey: segregation versus overlap of input zones from the primary motor cortex, the supplementary motor area, and the premotor cortex. Exp Brain Res 120:114–128CrossRefPubMed

Takara S, Hatanaka N, Takada M, Nambu A (2011) Differential activity patterns of putaminal neurons with inputs from the primary motor cortex and supplementary motor area in behaving monkeys. J Neurophysiol 106:1203–1217CrossRefPubMed

Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O (2002) Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142:284–291CrossRefPubMed

Takikawa Y, Kawagoe R, Hikosaka O (2004) A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping. J Neurophysiol 92:2520–2529CrossRefPubMed

Tanaka K (1996) Inferotemporal cortex and object vision. Annu Rev Neurosci 19:109–139CrossRefPubMed

Tepper JM, Martin LP, Anderson DR (1995) GABAA receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons. J Neurosci 15:3092–3103PubMed

Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12:97–136CrossRefPubMed

Tricomi E, Balleine BW, O’Doherty JP (2009) A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci 29:2225–2232CrossRefPubMedPubMedCentral

Van Hoesen GW, Yeterian EH, Lavizzo-Mourey R (1981) Widespread corticostriate projections from temporal cortex of the rhesus monkey. J Comp Neurol 199:205–219CrossRefPubMed

Waszcak BL, Walters JR (1983) Dopamine modulation of the effects of gamma-aminobutyric acid on substantia nigra pars reticulata neurons. Science 220:218–221CrossRefPubMed

Watanabe K, Lauwereyns J, Hikosaka O (2003) Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. J Neurosci 23:10052–10057PubMed

Wolfe JM (1994) Guided Search 2.0 A revised model of visual search. Psychon Bull Rev 1:202–238CrossRefPubMed

Wood W, Neal DT (2007) A new look at habits and the habit–goal interface. Psychol Rev 114:843–863CrossRefPubMed

Worbe Y, Baup N, Grabli D, Chaigneau M, Mounayar S, McCairn K, Feger J, Tremblay L (2009) Behavioral and movement disorders induced by local inhibitory dysfunction in primate striatum. Cereb Cortex 19:1844–1856CrossRefPubMed

Wunderlich K, Dayan P, Dolan RJ (2012) Mapping value based planning and extensively trained choice in the human brain. Nat Neurosci 15:786–791CrossRefPubMedPubMedCentral

Wymbs NF, Bassett DS, Mucha PJ, Porter MA, Grafton ST (2012) Differential recruitment of the sensorimotor putamen and frontoparietal cortex during motor chunking in humans. Neuron 74:936–946CrossRefPubMedPubMedCentral

Yamamoto S, Monosov IE, Yasuda M, Hikosaka O (2012) What and where information in the caudate tail guides saccades to visual objects. J Neurosci 32:11005–11016CrossRefPubMedPubMedCentral

Yamamoto S, Kim HF, Hikosaka O (2013) Reward value-contingent changes of visual responses in the primate caudate tail associated with a visuomotor skill. J Neurosci 33:11227–11238CrossRefPubMedPubMedCentral

Yasuda M, Hikosaka O (2015) Functional territories in primate substantia nigra pars reticulata separately signaling stable and flexible values. J Neurophysiol 113:1681–1696CrossRefPubMed

Yasuda M, Yamamoto S, Hikosaka O (2012) Robust representation of stable object values in the oculomotor basal ganglia. J Neurosci 32:16917–16932CrossRefPubMedPubMedCentral

Yin HH, Mulcare SP, Hilario MR, Clouse E, Holloway T, Davis MI, Hansson AC, Lovinger DM, Costa RM (2009) Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat Neurosci 12:333–341CrossRefPubMedPubMedCentral

Title: Parallel basal ganglia circuits for decision making
Authors: Okihide Hikosaka
Ali Ghazizadeh
Whitney Griggs
Hidetoshi Amita
Publication date: 01-03-2018
Publisher: Springer Vienna
Published in: Journal of Neural Transmission / Issue 3/2018
Print ISSN: 0300-9564
Electronic ISSN: 1435-1463
DOI: https://doi.org/10.1007/s00702-017-1691-1

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Parallel basal ganglia circuits for decision making

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 3/2018

The role of nonhuman primate models in the development of cell-based therapies for Parkinson’s disease

Dopamine and serotonin modulation of motor and non-motor functions of the non-human primate striato-pallidal circuits in normal and pathological states

Progress in developing transgenic monkey model for Huntington’s disease

The use of nonhuman primate models to understand processes in Parkinson’s disease

Reward and value coding by dopamine neurons in non-human primates

Non-human primate research of basal ganglia and movement disorders: advances and challenges