Skip to main content
Top
Published in: Journal of Neural Transmission 3/2018

Open Access 01-03-2018 | Neurology and Preclinical Neurological Studies - Review Article

Reward and value coding by dopamine neurons in non-human primates

Authors: Aydin Alikaya, Mackenzie Rack-Wildner, William R. Stauffer

Published in: Journal of Neural Transmission | Issue 3/2018

Login to get access

Abstract

Rewards are fundamental to everyday life. They confer pleasure, support learning, and mediate decisions. Dopamine-releasing neurons in the midbrain are critical for reward processing. These neurons receive input from more than 30 brain areas and send widespread projections to the basal ganglia and frontal cortex. Their phasic responses are tuned to rewards. Specifically, dopamine signals code reward prediction error, the difference between received and predicted rewards. Decades of research in awake, behaving non-human primates (NHP), have shown the importance of these neural signals for learning and decision making. In this review, we will provide an overview of the bedrock findings that support the reward prediction error hypothesis and examine evidence that this signal plays a role in learning and decision making. In addition, we will highlight some of the conceptual challenges in dopamine neurophysiology and identify future areas of research to address these challenges. Keeping with the theme of this special issue, we will focus on the role of NHP studies in understanding dopamine neurophysiology and make the argument that primate models are essential to this line of research.
Literature
go back to reference Aebischer P, Schultz W (1984) The activity of pars compacta neurons of the monkey substantia nigra is depressed by apomorphine. Neurosci Lett 50:25–29PubMedCrossRef Aebischer P, Schultz W (1984) The activity of pars compacta neurons of the monkey substantia nigra is depressed by apomorphine. Neurosci Lett 50:25–29PubMedCrossRef
go back to reference Alexander GE, DeLong MR (1985a) Microstimulation of the primate neostriatum. I. Physiological properties of striatal microexcitable zones. J Neurophysiol 53:1401–1416PubMedCrossRef Alexander GE, DeLong MR (1985a) Microstimulation of the primate neostriatum. I. Physiological properties of striatal microexcitable zones. J Neurophysiol 53:1401–1416PubMedCrossRef
go back to reference Alexander GE, DeLong MR (1985b) Microstimulation of the primate neostriatum. II. Somatotopic organization of striatal microexcitable zones and their relation to neuronal response properties. J Neurophysiol 53:1417–1430PubMedCrossRef Alexander GE, DeLong MR (1985b) Microstimulation of the primate neostriatum. II. Somatotopic organization of striatal microexcitable zones and their relation to neuronal response properties. J Neurophysiol 53:1417–1430PubMedCrossRef
go back to reference Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357–381PubMedCrossRef Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357–381PubMedCrossRef
go back to reference Bayer HM, Lau B, Glimcher PW (2007) Statistics of midbrain dopamine neuron spike trains in the awake primate. J Neurophysiol 98:1428–1439PubMedCrossRef Bayer HM, Lau B, Glimcher PW (2007) Statistics of midbrain dopamine neuron spike trains in the awake primate. J Neurophysiol 98:1428–1439PubMedCrossRef
go back to reference Boyden ES, Zhang F, Bamberg E, Nagel G, Deisseroth K (2005) Millisecond-timescale, genetically targeted optical control of neural activity. Nat Neurosci 8:1263–1268PubMedCrossRef Boyden ES, Zhang F, Bamberg E, Nagel G, Deisseroth K (2005) Millisecond-timescale, genetically targeted optical control of neural activity. Nat Neurosci 8:1263–1268PubMedCrossRef
go back to reference Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci 106:4894–4899PubMedPubMedCentralCrossRef Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci 106:4894–4899PubMedPubMedCentralCrossRef
go back to reference Bromberg-Martin ES, Matsumoto M, Hong S, Hikosaka O (2010) A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol 104:1068–1076PubMedPubMedCentralCrossRef Bromberg-Martin ES, Matsumoto M, Hong S, Hikosaka O (2010) A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol 104:1068–1076PubMedPubMedCentralCrossRef
go back to reference Bunney BS, Aghajanian GK, Roth RH (1973) Comparison of effects of l-dopa, amphetamine and apomorphine on firing rate of rat dopaminergic neurones. Nature 245:123–125 Bunney BS, Aghajanian GK, Roth RH (1973) Comparison of effects of l-dopa, amphetamine and apomorphine on firing rate of rat dopaminergic neurones. Nature 245:123–125
go back to reference Caraco T, Martindale S, Whittam TS (1980) An empirical demonstration of risk-sensitive foraging preferences. Anim Behav 28:820–830CrossRef Caraco T, Martindale S, Whittam TS (1980) An empirical demonstration of risk-sensitive foraging preferences. Anim Behav 28:820–830CrossRef
go back to reference Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N (2012) Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482:85–88PubMedPubMedCentralCrossRef Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N (2012) Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482:85–88PubMedPubMedCentralCrossRef
go back to reference Dahlstroem A, Fuxe K (1964) Evidence for the existence of monoamine-containing neurons in the central nervous system. I. Demonstration of monoamines in the cell bodies of brain stem neurons. Acta Physiol Scand Suppl 232:231–255 Dahlstroem A, Fuxe K (1964) Evidence for the existence of monoamine-containing neurons in the central nervous system. I. Demonstration of monoamines in the cell bodies of brain stem neurons. Acta Physiol Scand Suppl 232:231–255
go back to reference Debreu G (1959) Cardinal utility for even-chance mixtures of pairs of sure prospects. Rev Econ Stud 26:174–177CrossRef Debreu G (1959) Cardinal utility for even-chance mixtures of pairs of sure prospects. Rev Econ Stud 26:174–177CrossRef
go back to reference Dodson PD, Dreyer JK, Jennings KA, Syed ECJ, Wade-Martins R, Cragg SJ, Bolam JP, Magill PJ (2016) Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism. Proc Natl Acad Sci 113:E2180–E2188PubMedPubMedCentralCrossRef Dodson PD, Dreyer JK, Jennings KA, Syed ECJ, Wade-Martins R, Cragg SJ, Bolam JP, Magill PJ (2016) Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism. Proc Natl Acad Sci 113:E2180–E2188PubMedPubMedCentralCrossRef
go back to reference Eiselt AK, Nieder A (2013) Representation of abstract quantitative rules applied to spatial and numerical magnitudes in primate prefrontal cortex. J Neurosci 33:7526–7534PubMedCrossRef Eiselt AK, Nieder A (2013) Representation of abstract quantitative rules applied to spatial and numerical magnitudes in primate prefrontal cortex. J Neurosci 33:7526–7534PubMedCrossRef
go back to reference El-Shamayleh Y, Kojima Y, Soetedjo R, Horwitz GD (2017) Selective optogenetic control of Purkinje cells in monkey cerebellum. Neuron 95:51–62PubMedCrossRef El-Shamayleh Y, Kojima Y, Soetedjo R, Horwitz GD (2017) Selective optogenetic control of Purkinje cells in monkey cerebellum. Neuron 95:51–62PubMedCrossRef
go back to reference Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M (2011) Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA 108:15462–15467PubMedPubMedCentralCrossRef Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M (2011) Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA 108:15462–15467PubMedPubMedCentralCrossRef
go back to reference Eshel N, Bukwich M, Rao V, Hemmelder V, Tian J, Uchida N (2015) Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525:243–246PubMedPubMedCentralCrossRef Eshel N, Bukwich M, Rao V, Hemmelder V, Tian J, Uchida N (2015) Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525:243–246PubMedPubMedCentralCrossRef
go back to reference Fiorillo CD (2013) Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341:546–549PubMedCrossRef Fiorillo CD (2013) Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341:546–549PubMedCrossRef
go back to reference Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902PubMedCrossRef Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902PubMedCrossRef
go back to reference Fiorillo CD, Newsome WT, Schultz W (2008) The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 11:966–973PubMedCrossRef Fiorillo CD, Newsome WT, Schultz W (2008) The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 11:966–973PubMedCrossRef
go back to reference Fiorillo CD, Song MR, Yun SR (2013a) Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 33:4710–4725PubMedCrossRef Fiorillo CD, Song MR, Yun SR (2013a) Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 33:4710–4725PubMedCrossRef
go back to reference Fiorillo CD, Yun SR, Song MR (2013b) Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci 33:4693–4709PubMedCrossRef Fiorillo CD, Yun SR, Song MR (2013b) Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci 33:4693–4709PubMedCrossRef
go back to reference Galvan A, Caiola MJ, Albaugh DL (2017) Advances in optogenetic and chemogenetic methods to study brain circuits in non-human primates. J Neural Transm (Vienna). doi:10.1007/s00702-017-1697-8 Galvan A, Caiola MJ, Albaugh DL (2017) Advances in optogenetic and chemogenetic methods to study brain circuits in non-human primates. J Neural Transm (Vienna). doi:10.​1007/​s00702-017-1697-8
go back to reference Genest W, Stauffer WR, Schultz W (2016) Utility functions predict variance and skewness risk preferences in monkeys. Proc Natl Acad Sci USA 113:8402–8407PubMedPubMedCentralCrossRef Genest W, Stauffer WR, Schultz W (2016) Utility functions predict variance and skewness risk preferences in monkeys. Proc Natl Acad Sci USA 113:8402–8407PubMedPubMedCentralCrossRef
go back to reference German DC, Manaye KF (1993) Midbrain dopaminergic neurons (nuclei A8, A9, and A10): three-dimensional reconstruction in the rat. J Comp Neurol 331:297–309PubMedCrossRef German DC, Manaye KF (1993) Midbrain dopaminergic neurons (nuclei A8, A9, and A10): three-dimensional reconstruction in the rat. J Comp Neurol 331:297–309PubMedCrossRef
go back to reference Gonon FG (1988) Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry. Neuroscience 24:19–28PubMedCrossRef Gonon FG (1988) Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry. Neuroscience 24:19–28PubMedCrossRef
go back to reference Grace AA, Bunney BS (1983) Intracellular and extracellular electrophysiology of nigral dopaminergic neurons—1. Identification and characterization. Neuroscience 10:301–315PubMedCrossRef Grace AA, Bunney BS (1983) Intracellular and extracellular electrophysiology of nigral dopaminergic neurons—1. Identification and characterization. Neuroscience 10:301–315PubMedCrossRef
go back to reference Guyenet PG, Aghajanian GK (1978) Antidromic identification of dopaminergic and other output neurons of the rat substantia nigra. Brain Res 150:69–84PubMedCrossRef Guyenet PG, Aghajanian GK (1978) Antidromic identification of dopaminergic and other output neurons of the rat substantia nigra. Brain Res 150:69–84PubMedCrossRef
go back to reference Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD (2016) Mesolimbic dopamine signals the value of work. Nat Neurosci 19:117–126PubMedCrossRef Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD (2016) Mesolimbic dopamine signals the value of work. Nat Neurosci 19:117–126PubMedCrossRef
go back to reference Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304–309PubMedCrossRef Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304–309PubMedCrossRef
go back to reference Howe MW, Tierney PL, Sandberg SG, Phillips PE, Graybiel AM (2013) Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500:575–579PubMedPubMedCentralCrossRef Howe MW, Tierney PL, Sandberg SG, Phillips PE, Graybiel AM (2013) Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500:575–579PubMedPubMedCentralCrossRef
go back to reference Jacob SN, Ott T, Nieder A (2013) Dopamine regulates two classes of primate prefrontal neurons that represent sensory signals. J Neurosci 33:13724–13734PubMedCrossRef Jacob SN, Ott T, Nieder A (2013) Dopamine regulates two classes of primate prefrontal neurons that represent sensory signals. J Neurosci 33:13724–13734PubMedCrossRef
go back to reference Jacob SN, Stalter M, Nieder A (2016) Cell-type-specific modulation of targets and distractors by dopamine D1 receptors in primate prefrontal cortex. Nat Commun 7:13218PubMedPubMedCentralCrossRef Jacob SN, Stalter M, Nieder A (2016) Cell-type-specific modulation of targets and distractors by dopamine D1 receptors in primate prefrontal cortex. Nat Commun 7:13218PubMedPubMedCentralCrossRef
go back to reference Klein C, Evrard HC, Shapcott KA, Haverkamp S, Logothetis NK, Schmid MC (2016) Cell-targeted optogenetics and electrical microstimulation reveal the primate koniocellular projection to supra-granular visual cortex. Neuron 90:143–151PubMedCrossRef Klein C, Evrard HC, Shapcott KA, Haverkamp S, Logothetis NK, Schmid MC (2016) Cell-targeted optogenetics and electrical microstimulation reveal the primate koniocellular projection to supra-granular visual cortex. Neuron 90:143–151PubMedCrossRef
go back to reference Lak A, Stauffer WR, Schultz W (2014) Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA 111:2343–2348PubMedPubMedCentralCrossRef Lak A, Stauffer WR, Schultz W (2014) Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA 111:2343–2348PubMedPubMedCentralCrossRef
go back to reference Lak A, Nomoto K, Keramati M, Sakagami M, Kepecs A (2017) Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision. Curr Biol 27:821–832PubMedCrossRef Lak A, Nomoto K, Keramati M, Sakagami M, Kepecs A (2017) Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision. Curr Biol 27:821–832PubMedCrossRef
go back to reference Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC (2012) Input-specific control of reward and aversion in the ventral tegmental area. Nature 491:212–217PubMedPubMedCentralCrossRef Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC (2012) Input-specific control of reward and aversion in the ventral tegmental area. Nature 491:212–217PubMedPubMedCentralCrossRef
go back to reference Lammel S, Steinberg EE, Foldy C, Wall NR, Beier K, Luo L, Malenka RC (2015) Diversity of transgenic mouse models for selective targeting of midbrain dopamine neurons. Neuron 85:429–438PubMedPubMedCentralCrossRef Lammel S, Steinberg EE, Foldy C, Wall NR, Beier K, Luo L, Malenka RC (2015) Diversity of transgenic mouse models for selective targeting of midbrain dopamine neurons. Neuron 85:429–438PubMedPubMedCentralCrossRef
go back to reference Langston JW, Ballard P, Tetrud JW, Irwin I (1983) Chronic Parkinsonism in humans due to a product of meperidine-analog synthesis. Science 219:979–980PubMedCrossRef Langston JW, Ballard P, Tetrud JW, Irwin I (1983) Chronic Parkinsonism in humans due to a product of meperidine-analog synthesis. Science 219:979–980PubMedCrossRef
go back to reference Lewis DA, Campbell MJ, Foote SL, Goldstein M, Morrison JH (1987) The distribution of tyrosine hydroxylase-immunoreactive fibers in primate neocortex is widespread but regionally specific. J Neurosci 7:279–290PubMed Lewis DA, Campbell MJ, Foote SL, Goldstein M, Morrison JH (1987) The distribution of tyrosine hydroxylase-immunoreactive fibers in primate neocortex is widespread but regionally specific. J Neurosci 7:279–290PubMed
go back to reference Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145–163PubMedCrossRef Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145–163PubMedCrossRef
go back to reference Lovenberg W, Weissbach H, Udenfriend S (1962) Aromatic l-amino acid decarboxylase. J Biol Chem 237:89–93PubMed Lovenberg W, Weissbach H, Udenfriend S (1962) Aromatic l-amino acid decarboxylase. J Biol Chem 237:89–93PubMed
go back to reference Lynd-Balta E, Haber SN (1994a) The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum. Neuroscience 59:625–640PubMedCrossRef Lynd-Balta E, Haber SN (1994a) The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum. Neuroscience 59:625–640PubMedCrossRef
go back to reference Lynd-Balta E, Haber SN (1994b) The organization of midbrain projections to the ventral striatum in the primate. Neuroscience 59:609–623PubMedCrossRef Lynd-Balta E, Haber SN (1994b) The organization of midbrain projections to the ventral striatum in the primate. Neuroscience 59:609–623PubMedCrossRef
go back to reference Lynd-Balta E, Haber SN (1994c) Primate striatonigral projections: a comparison of the sensorimotor-related striatum and the ventral striatum. J Comp Neurol 345:562–578PubMedCrossRef Lynd-Balta E, Haber SN (1994c) Primate striatonigral projections: a comparison of the sensorimotor-related striatum and the ventral striatum. J Comp Neurol 345:562–578PubMedCrossRef
go back to reference Machina MJ (1987) Choice under uncertainty: problems solved and unsolved. J Econ Perspect 1:121–154CrossRef Machina MJ (1987) Choice under uncertainty: problems solved and unsolved. J Econ Perspect 1:121–154CrossRef
go back to reference McCoy AN, Platt ML (2005) Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci 8:1220–1227PubMedCrossRef McCoy AN, Platt ML (2005) Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci 8:1220–1227PubMedCrossRef
go back to reference Min HK, Ross EK, Jo HJ, Cho S, Settell ML, Jeong JH, Duffy PS, Chang SY, Bennet KE, Blaha CD et al (2016) Dopamine release in the nonhuman primate caudate and putamen depends upon site of stimulation in the subthalamic nucleus. J Neurosci 36:6022–6029PubMedPubMedCentralCrossRef Min HK, Ross EK, Jo HJ, Cho S, Settell ML, Jeong JH, Duffy PS, Chang SY, Bennet KE, Blaha CD et al (2016) Dopamine release in the nonhuman primate caudate and putamen depends upon site of stimulation in the subthalamic nucleus. J Neurosci 36:6022–6029PubMedPubMedCentralCrossRef
go back to reference Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024–1027PubMedCrossRef Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024–1027PubMedCrossRef
go back to reference Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451PubMedCrossRef Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451PubMedCrossRef
go back to reference Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947PubMed Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947PubMed
go back to reference Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H (2004) Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43:133–143PubMedCrossRef Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H (2004) Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43:133–143PubMedCrossRef
go back to reference Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9:1057–1063PubMedCrossRef Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9:1057–1063PubMedCrossRef
go back to reference Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280PubMedCrossRef Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280PubMedCrossRef
go back to reference Nomoto K, Schultz W, Watanabe T, Sakagami M (2010) Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci 30:10692–10702PubMedPubMedCentralCrossRef Nomoto K, Schultz W, Watanabe T, Sakagami M (2010) Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci 30:10692–10702PubMedPubMedCentralCrossRef
go back to reference Noudoost B, Moore T (2011b) A reliable microinjectrode system for use in behaving monkeys. J Neurosci Methods 194:218–223PubMedCrossRef Noudoost B, Moore T (2011b) A reliable microinjectrode system for use in behaving monkeys. J Neurosci Methods 194:218–223PubMedCrossRef
go back to reference Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419–427PubMedCrossRef Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419–427PubMedCrossRef
go back to reference O’Neill M, Schultz W (2010) Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68:789–800PubMedCrossRef O’Neill M, Schultz W (2010) Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68:789–800PubMedCrossRef
go back to reference Pan WX, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235–6242PubMedCrossRef Pan WX, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235–6242PubMedCrossRef
go back to reference Puig MV, Miller EK (2012) The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning. Neuron 74:874–886PubMedCrossRef Puig MV, Miller EK (2012) The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning. Neuron 74:874–886PubMedCrossRef
go back to reference Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non reinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64–99 Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non reinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64–99
go back to reference Romo R, Schultz W (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63:592–606PubMedCrossRef Romo R, Schultz W (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63:592–606PubMedCrossRef
go back to reference Schultz W (1986) Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56:1439–1461PubMedCrossRef Schultz W (1986) Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56:1439–1461PubMedCrossRef
go back to reference Schultz W, Romo R (1987) Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol 57:201–217PubMedCrossRef Schultz W, Romo R (1987) Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol 57:201–217PubMedCrossRef
go back to reference Schultz W, Romo R (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63:607–624PubMedCrossRef Schultz W, Romo R (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63:607–624PubMedCrossRef
go back to reference Schultz W, Ruffieux A, Aebischer P (1983) The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res 51:377–387CrossRef Schultz W, Ruffieux A, Aebischer P (1983) The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res 51:377–387CrossRef
go back to reference Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13:900–913PubMed Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13:900–913PubMed
go back to reference Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599PubMedCrossRef Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599PubMedCrossRef
go back to reference Smiley JF, Levey AI, Ciliax BJ, Goldman-Rakic PS (1994) D1 dopamine receptor immunoreactivity in human and monkey cerebral cortex: predominant and extrasynaptic localization in dendritic spines. Proc Natl Acad Sci USA 91:5720–5724PubMedPubMedCentralCrossRef Smiley JF, Levey AI, Ciliax BJ, Goldman-Rakic PS (1994) D1 dopamine receptor immunoreactivity in human and monkey cerebral cortex: predominant and extrasynaptic localization in dendritic spines. Proc Natl Acad Sci USA 91:5720–5724PubMedPubMedCentralCrossRef
go back to reference Smith Y, Wichmann T, DeLong MR (2014) Corticostriatal and mesocortical dopamine systems: do species differences matter? Nat Rev Neurosci 15:63PubMedCrossRef Smith Y, Wichmann T, DeLong MR (2014) Corticostriatal and mesocortical dopamine systems: do species differences matter? Nat Rev Neurosci 15:63PubMedCrossRef
go back to reference Stark AK, Pakkenberg B (2004) Histological changes of the dopaminergic nigrostriatal system in aging. Cell Tissue Res 318:81–92PubMedCrossRef Stark AK, Pakkenberg B (2004) Histological changes of the dopaminergic nigrostriatal system in aging. Cell Tissue Res 318:81–92PubMedCrossRef
go back to reference Stauffer WR, Lak A, Yang A, Borel M, Paulsen O, Boyden ES, Schultz W (2016) Dopamine neuron-specific optogenetic stimulation in rhesus macaques. Cell 166(1564–1571):e1566 Stauffer WR, Lak A, Yang A, Borel M, Paulsen O, Boyden ES, Schultz W (2016) Dopamine neuron-specific optogenetic stimulation in rhesus macaques. Cell 166(1564–1571):e1566
go back to reference Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH (2013) A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16:966–973PubMedPubMedCentralCrossRef Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH (2013) A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16:966–973PubMedPubMedCentralCrossRef
go back to reference Studer A, Schultz W (1987) The catecholamine uptake inhibitor nomifensine depresses impulse activity of dopamine neurons in mouse substantia nigra. Neurosci Lett 80:207–212PubMedCrossRef Studer A, Schultz W (1987) The catecholamine uptake inhibitor nomifensine depresses impulse activity of dopamine neurons in mouse substantia nigra. Neurosci Lett 80:207–212PubMedCrossRef
go back to reference Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press, Cambridge Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
go back to reference Threlfell S, Lalic T, Platt NJ, Jennings KA, Deisseroth K, Cragg SJ (2012) Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron 75:58–64PubMedCrossRef Threlfell S, Lalic T, Platt NJ, Jennings KA, Deisseroth K, Cragg SJ (2012) Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron 75:58–64PubMedCrossRef
go back to reference Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645PubMedCrossRef Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645PubMedCrossRef
go back to reference Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324:1080–1084PubMedPubMedCentralCrossRef Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324:1080–1084PubMedPubMedCentralCrossRef
go back to reference Ungless MA, Grace AA (2012) Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci 35:422–430PubMedPubMedCentralCrossRef Ungless MA, Grace AA (2012) Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci 35:422–430PubMedPubMedCentralCrossRef
go back to reference Vijayraghavan S, Wang M, Birnbaum SG, Williams GV, Arnsten AF (2007) Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory. Nat Neurosci 10:376–384PubMedCrossRef Vijayraghavan S, Wang M, Birnbaum SG, Williams GV, Arnsten AF (2007) Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory. Nat Neurosci 10:376–384PubMedCrossRef
go back to reference von Neumann J, Morgenstern O, Kuhn HW, Rubinstein A (1944) Theory of games and economic behavior (60th anniversary commemorative edition). Princeton University Press, Princeton von Neumann J, Morgenstern O, Kuhn HW, Rubinstein A (1944) Theory of games and economic behavior (60th anniversary commemorative edition). Princeton University Press, Princeton
go back to reference Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43–48PubMedCrossRef Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43–48PubMedCrossRef
go back to reference Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N (2012) Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74:858–873PubMedCrossRef Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N (2012) Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74:858–873PubMedCrossRef
go back to reference Williams SM, Goldman-Rakic PS (1993) Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cereb Cortex 3:199–222PubMedCrossRef Williams SM, Goldman-Rakic PS (1993) Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cereb Cortex 3:199–222PubMedCrossRef
go back to reference Williams GV, Goldman-Rakic PS (1995) Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature 376:572–575PubMedCrossRef Williams GV, Goldman-Rakic PS (1995) Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature 376:572–575PubMedCrossRef
go back to reference Williams SM, Goldman-Rakic PS (1998) Widespread origin of the primate mesofrontal dopamine system. Cereb Cortex 8:321–345PubMedCrossRef Williams SM, Goldman-Rakic PS (1998) Widespread origin of the primate mesofrontal dopamine system. Cereb Cortex 8:321–345PubMedCrossRef
go back to reference Yamada H, Tymula A, Louie K, Glimcher PW (2013) Thirst-dependent risk preferences in monkeys identify a primitive form of wealth. Proc Natl Acad Sci USA 110:15788–15793PubMedPubMedCentralCrossRef Yamada H, Tymula A, Louie K, Glimcher PW (2013) Thirst-dependent risk preferences in monkeys identify a primitive form of wealth. Proc Natl Acad Sci USA 110:15788–15793PubMedPubMedCentralCrossRef
go back to reference Yazdan-Shahmorad A, Diaz-Botia C, Hanson TL, Kharazia V, Ledochowitsch P, Maharbiz MM, Sabes PN (2016) A large-scale interface for optogenetic stimulation and recording in nonhuman primates. Neuron 89:927–939PubMedCrossRef Yazdan-Shahmorad A, Diaz-Botia C, Hanson TL, Kharazia V, Ledochowitsch P, Maharbiz MM, Sabes PN (2016) A large-scale interface for optogenetic stimulation and recording in nonhuman primates. Neuron 89:927–939PubMedCrossRef
go back to reference Yelin R, Schuldiner S (1995) The pharmacological profile of the vesicular monoamine transporter resembles that of multidrug transporters. FEBS Lett 377:201–207PubMedCrossRef Yelin R, Schuldiner S (1995) The pharmacological profile of the vesicular monoamine transporter resembles that of multidrug transporters. FEBS Lett 377:201–207PubMedCrossRef
go back to reference Yoshimi K, Kumada S, Weitemier A, Jo T, Inoue M (2015) Reward-induced phasic dopamine release in the monkey ventral striatum and putamen. PLoS One 10:e0130443PubMedPubMedCentralCrossRef Yoshimi K, Kumada S, Weitemier A, Jo T, Inoue M (2015) Reward-induced phasic dopamine release in the monkey ventral striatum and putamen. PLoS One 10:e0130443PubMedPubMedCentralCrossRef
Metadata
Title
Reward and value coding by dopamine neurons in non-human primates
Authors
Aydin Alikaya
Mackenzie Rack-Wildner
William R. Stauffer
Publication date
01-03-2018
Publisher
Springer Vienna
Published in
Journal of Neural Transmission / Issue 3/2018
Print ISSN: 0300-9564
Electronic ISSN: 1435-1463
DOI
https://doi.org/10.1007/s00702-017-1793-9

Other articles of this Issue 3/2018

Journal of Neural Transmission 3/2018 Go to the issue

Neurology and Preclinical Neurological Studies - Review Article

Advances in optogenetic and chemogenetic methods to study brain circuits in non-human primates

Neurology and Preclinical Neurological Studies - Review Article

Parallel basal ganglia circuits for decision making

Neurology and Preclinical Neurological Studies - Review Article

α-Synuclein nonhuman primate models of Parkinson’s disease