Access the full text.
Sign up today, get DeepDyve free for 14 days.
A. Dickinson (2012)
Associative learning and animal cognitionPhilosophical Transactions of the Royal Society B: Biological Sciences, 367
W. Schultz, P. Apicella, E. Scarnati, Tomas Ljungbergc (1992)
Neuronal activity in monkey ventral striatum related to the expectation of reward, 12
H. Yin, Sean Ostlund, B. Knowlton, B. Balleine (2005)
The role of the dorsomedial striatum in instrumental conditioningEuropean Journal of Neuroscience, 22
J. Hagan, Jonathan Alpert, R. Morris, S. Iversen (1983)
The effects of central catecholamine depletions on spatial learning in ratsBehavioural Brain Research, 9
J. Elman (1990)
Finding Structure in TimeCogn. Sci., 14
R. Cardinal, J. Parkinson, Jeremy Hall, B. Everitt (2002)
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortexNeuroscience & Biobehavioral Reviews, 26
A. Faure, U. Haberland, F. Condé, N. Massioui (2005)
Lesion to the Nigrostriatal Dopamine System Disrupts Stimulus-Response Habit FormationThe Journal of Neuroscience, 25
E. Baeg, Yunbok Kim, K. Huh, I. Mook-Jung, Hyun Kim, M. Jung (2003)
Dynamics of Population Code for Working Memory in the Prefrontal CortexNeuron, 40
Adam Steiner, David Redish (2014)
Behavioral and neurophysiological correlates of regret in rat decision-making on a neuroeconomic taskNature neuroscience, 17
James Young, M. Shapiro (2011)
Dynamic Coding of Goal-Directed Paths by Orbital Prefrontal CortexThe Journal of Neuroscience, 31
David Badre, D. Nee (2018)
Frontal Cortex and the Hierarchical Control of BehaviorTrends in Cognitive Sciences, 22
Wendy Inglis, P. Winn (1995)
The pedunculopontine tegmental nucleus: Where the striatum meets the reticular formationProgress in Neurobiology, 47
Peter Redgrave, Tony Prescott, Kevin Gurney (1999)
The basal ganglia: a vertebrate solution to the selection problem?Neuroscience, 89
H. Cabral, M. Vinck, C. Fouquet, C. Pennartz, L. Rondi-Reig, F. Battaglia (2014)
Oscillatory Dynamics and Place Field Maps Reflect Hippocampal Ensemble Processing of Sequence and Place Memory under NMDA Receptor ControlNeuron, 81
S. Corkin (2002)
What's new with the amnesic patient H.M.?Nature Reviews Neuroscience, 3
T. Robbins, M. Cador, JaneR Taylor, B. Everitt (1989)
Limbic-striatal interactions in reward-related processesNeuroscience & Biobehavioral Reviews, 13
A. Luthi (2014)
Sleep spindles: Where they come from, what they do, 20
C. Pennartz, H. Groenewegen, F. Silva (1994)
The nucleus accumbens as a complex of functionally distinct neuronal ensembles: An integration of behavioural, electrophysiological and anatomical dataProgress in Neurobiology, 42
G. Pezzulo, M. Meer, C. Lansink, C. Pennartz (2014)
Internally generated sequences in learning and executing goal-directed behaviorTrends in Cognitive Sciences, 18
A. Bari, T. Robbins (2013)
Inhibition and impulsivity: Behavioral and neural basis of response controlProgress in Neurobiology, 108
Brandy Schmidt, Anneke Duin, A. Redish (2019)
Disrupting the medial prefrontal cortex alters hippocampal sequences during deliberative decision making.Journal of neurophysiology, 121 6
F. Sato, P. Lavallée, M. Lévesque, A. Parent (2000)
Single‐axon tracing study of neurons of the external segment of the globus pallidus in primateJournal of Comparative Neurology, 417
I. Bar-Gad, Genela Morris, H. Bergman (2003)
Information processing, dimensionality reduction and reinforcement learning in the basal gangliaProgress in Neurobiology, 71
Jennifer Birrell, V. Brown (2000)
Medial Frontal Cortex Mediates Perceptual Attentional Set Shifting in the RatThe Journal of Neuroscience, 20
Wenbo Tang, Justin Shin, L. Frank, Shantanu Jadhav (2017)
Hippocampal-Prefrontal Reactivation during Learning Is Stronger in Awake Compared with Sleep StatesThe Journal of Neuroscience, 37
C. Padoa-Schioppa, Katherine Conen (2017)
Orbitofrontal Cortex: A Neural Circuit for Economic DecisionsNeuron, 96
Wei Wei, Xiao-Jing Wang (2016)
Inhibitory Control in the Cortico-Basal Ganglia-Thalamocortical Loop: Complex Regulation and Interplay with Memory and Decision ProcessesNeuron, 92
S. Haber, J. Fudge, Nikolaus McFarland (2000)
Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral StriatumThe Journal of Neuroscience, 20
C. Pennartz, R. Ameerun, H. Groenewegen, F. Silva (1993)
Synaptic Plasticity in an In Vitro Slice Preparation of the Rat Nucleus AccumbensEuropean Journal of Neuroscience, 5
M. Botvinick (2008)
Hierarchical models of behavior and prefrontal functionTrends in Cognitive Sciences, 12
H. Eichenbaum (2017)
Prefrontal–hippocampal interactions in episodic memoryNature Reviews Neuroscience, 18
J. O’Keefe, J. Dostrovsky (1971)
The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat.Brain research, 34 1
N. Eshel, Michael Bukwich, Vinod Rao, Vivian Hemmelder, Ju Tian, N. Uchida (2015)
Arithmetic and local circuitry underlying dopamine prediction errorsNature, 525
Kara Agster, N. Fortin, H. Eichenbaum (2002)
The Hippocampus and Disambiguation of Overlapping SequencesThe Journal of Neuroscience, 22
J. C. Houk, J. L. Adams, A. G. Barto (1995)
Models of information processing in the basal ganglia
P. Goltstein, Guido Meijer, C. Pennartz (2018)
Conditioning sharpens the spatial representation of rewarded stimuli in mouse primary visual cortexeLife, 7
Jenia Bugaysen, I. Bar-Gad, A. Korngreen (2013)
Continuous Modulation of Action Potential Firing by a Unitary GABAergic Connection in the Globus Pallidus In VitroThe Journal of Neuroscience, 33
K. Doya (1999)
What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?Neural networks : the official journal of the International Neural Network Society, 12 7-8
C. Pennartz, Eunjeong Lee, J. Verheul, P. Lipa, C. Barnes, B. McNaughton (2004)
The Ventral Striatum in Off-Line Processing: Ensemble Reactivation during Sleep and Modulation by Hippocampal RipplesThe Journal of Neuroscience, 24
Dylan Simon, N. Daw (2011)
Neural Correlates of Forward Planning in a Spatial Decision Task in HumansThe Journal of Neuroscience, 31
P. Voorn, L. Vanderschuren, H. Groenewegen, T. Robbins, C. Pennartz (2004)
Putting a spin on the dorsal–ventral divide of the striatumTrends in Neurosciences, 27
(2007)
Schemas and memory consolidationScience (New York, N.Y.), 316
J. Huguenard, D. McCormick (2007)
Thalamic synchrony and dynamic regulation of global forebrain oscillationsTrends in Neurosciences, 30
V. Hok, E. Save, P. Lenck-Santini, B. Poucet (2005)
Coding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex.Proceedings of the National Academy of Sciences of the United States of America, 102 12
Andrew Wikenheiser, David Redish (2015)
Hippocampal theta sequences reflect current goalsNature Neuroscience, 18
A. Kelley (2004)
Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learningNeuroscience & Biobehavioral Reviews, 27
H. Uylings, H. Groenewegen, B. Kolb (2003)
Do rats have a prefrontal cortex?Behavioural Brain Research, 146
M. Meer, Adam Johnson, Adam Johnson, N. Schmitzer-Torbert, N. Schmitzer-Torbert, A. Redish (2010)
Triple Dissociation of Information Processing in Dorsal Striatum, Ventral Striatum, and Hippocampus on a Learned Spatial Decision TaskNeuron, 67
M. Meer, A. Redish (2011)
Ventral striatum: a critical look at models of learning and evaluationCurrent Opinion in Neurobiology, 21
Adam Johnson, A. Redish (2007)
Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision PointThe Journal of Neuroscience, 27
J. Wallis, K. Anderson, E. Miller (2001)
Single neurons in prefrontal cortex encode abstract rulesNature, 411
G. Buzsáki, E. Moser (2013)
Memory, navigation and theta rhythm in the hippocampal-entorhinal systemNature Neuroscience, 16
R. A. Rescorla, A. R. Wagner (1972)
Classical conditioning II: Current research
C. Pennartz (2018)
Consciousness, Representation, Action: The Importance of Being Goal-DirectedTrends in Cognitive Sciences, 22
Gabrielle Girardeau, Ingrid Inema, G. Buzsáki (2017)
Reactivations of emotional memory in the hippocampus–amygdala system during sleepNature Neuroscience, 20
Thomas Davidson, F. Kloosterman, M. Wilson (2009)
Hippocampal Replay of Extended ExperienceNeuron, 63
Axel Cleeremans, A. Destrebecqz, Maud Boyer (1998)
Implicit learning: news from the frontTrends in Cognitive Sciences, 2
G. Hart, BeatriceK Leung, B. Balleine (2014)
Dorsal and ventral streams: The distinct role of striatal subregions in the acquisition and performance of goal-directed actionsNeurobiology of Learning and Memory, 108
D. Durstewitz, Nicole Vittoz, S. Floresco, J. Seamans (2010)
Abrupt Transitions between Prefrontal Neural Ensemble States Accompany Behavioral Transitions during Rule LearningNeuron, 66
D. Plenz (2003)
When inhibition goes incognito: feedback interaction between spiny projection neurons in striatal functionTrends in Neurosciences, 26
Warren Meck (1988)
Hippocampal function is required for feedback control of an internal clock's criterion.Behavioral neuroscience, 102 1
C. MacDonald, Stephen Carrow, Ryan Place, H. Eichenbaum (2013)
Distinct Hippocampal Time Cell Sequences Represent Odor Memories in Immobilized RatsThe Journal of Neuroscience, 33
Andrew Papale, Mark Zielinski, L. Frank, Shantanu Jadhav, A. Redish (2016)
Interplay between Hippocampal Sharp-Wave-Ripple Events and Vicarious Trial and Error Behaviors in Decision MakingNeuron, 92
T. Robbins, B. Everitt (1982)
Functional studies of the central catecholamines.International review of neurobiology, 23
Sarah Heilbronner, J. Rodriguez-Romaguera, G. Quirk, H. Groenewegen, S. Haber (2016)
Circuit-Based Corticostriatal Homologies Between Rat and PrimateBiological Psychiatry, 80
J. Huxter, N. Burgess, J. O’Keefe (2003)
Independent rate and temporal coding in hippocampal pyramidal cellsNature, 425
F. P. Battaglia, C. M. Pennartz (2011)
The construction of semantic memory: Grammar‐based representations learned from relational episodic information, 5
P. Redgrave, T. Prescott, K. Gurney (1999)
Is the short-latency dopamine response too short to signal reward error?Trends in Neurosciences, 22
A. Barto, S. Mahadevan (2003)
Recent Advances in Hierarchical Reinforcement LearningDiscrete Event Dynamic Systems, 13
C. Harvey, P. Coen, D. Tank (2012)
Choice-specific sequences in parietal cortex during a virtual-navigation decision taskNature, 484
Georg Keller, T. Bonhoeffer, M. Hübener (2012)
Sensorimotor Mismatch Signals in Primary Visual Cortex of the Behaving MouseNeuron, 74
L. Corbit, J. Muir, B. Balleine (2003)
Lesions of mediodorsal thalamus and anterior thalamic nuclei produce dissociable effects on instrumental conditioning in ratsEuropean Journal of Neuroscience, 18
H. Kudrimoti, C. Barnes, B. McNaughton (1999)
Reactivation of Hippocampal Cell Assemblies: Effects of Behavioral State, Experience, and EEG DynamicsThe Journal of Neuroscience, 19
M. Shuler, M. Bear (2006)
Reward Timing in the Primary Visual CortexScience, 311
Matthew Jones, M. Wilson (2005)
Theta Rhythms Coordinate Hippocampal–Prefrontal Interactions in a Spatial Memory TaskPLoS Biology, 3
M. Wingerden, M. Vinck, J. Lankelma, C. Pennartz, M. Wingerden, M. Vinck, J. Lankelma, C. Pennartz
Source (or Part of the following Source): Type Article Title Learning-associated Gamma-band Phase-locking of Action-outcome Selective Neurons in Orbitofrontal Cortex Author(s) Learning-associated Gamma-band Phase-locking of Action–outcome Selective Neurons in Orbitofrontal Cortex Gamma Oscillations
Daisuke Miyamoto, D. Hirai, C. Fung, Ayumu Inutsuka, Maya Odagawa, T. Suzuki, R. Boehringer, Chinnakkaruppan Adaikkan, Chie Matsubara, N. Matsuki, T. Fukai, T. McHugh, A. Yamanaka, Masanori Murayama (2016)
Top-down cortical input during NREM sleep consolidates perceptual memoryScience, 352
M. Roitman, R. Wheeler, R. Carelli (2005)
Nucleus Accumbens Neurons Are Innately Tuned for Rewarding and Aversive Taste Stimuli, Encode Their Predictors, and Are Linked to Motor OutputNeuron, 45
(1951)
Hixon symposium on cerebral mechanisms in behavior
T. Stalnaker, Nisha Cooch, G. Schoenbaum (2015)
What the orbitofrontal cortex does not doNature Neuroscience, 18
M. Rodriguez-Oroz, M. Jahanshahi, P. Krack, I. Litvan, R. Macias, E. Bézard, J. Obeso (2009)
Initial clinical manifestations of Parkinson's disease: features and pathophysiological mechanismsThe Lancet Neurology, 8
C. Pennartz (1995)
The ascending neuromodulatory systems in learning by reinforcement: comparing computational conjectures with experimental findingsBrain Research Reviews, 21
W. Penny, P. Zeidman, N. Burgess (2013)
Forward and Backward Inference in Spatial CognitionPLoS Computational Biology, 9
HW Berendse, H. Groenewegen, A. Lohman (1992)
Compartmental distribution of ventral striatal neurons projecting to the mesencephalon in the rat, 12
N. Dembrow, B. Zemelman, D. Johnston (2015)
Temporal Dynamics of L5 Dendrites in Medial Prefrontal Cortex Regulate Integration Versus Coincidence Detection of Afferent InputsThe Journal of Neuroscience, 35
A. Dezfouli, Nura Lingawi, B. Balleine (2014)
Habits as action sequences: hierarchical action control and changes in outcome valuePhilosophical Transactions of the Royal Society B: Biological Sciences, 369
M. Rushworth, M. Noonan, E. Boorman, M. Walton, T. Behrens (2011)
Frontal Cortex and Reward-Guided Learning and Decision-MakingNeuron, 70
W. Kargo, B. Szatmáry, D. Nitz (2007)
Adaptation of Prefrontal Cortical Firing Patterns and Their Fidelity to Changes in Action–Reward ContingenciesThe Journal of Neuroscience, 27
J. Gläscher, R. Adolphs, H. Damasio, A. Bechara, D. Rudrauf, M. Calamia, L. Paul, D. Tranel (2012)
Lesion mapping of cognitive control and value-based decision making in the prefrontal cortexProceedings of the National Academy of Sciences, 109
H. Yin, B. Knowlton, B. Balleine (2004)
Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learningEuropean Journal of Neuroscience, 19
A. Mar, Alice Walker, D. Theobald, D. Eagle, T. Robbins (2011)
Dissociable Effects of Lesions to Orbitofrontal Cortex Subregions on Impulsive Choice in the RatThe Journal of Neuroscience, 31
J. Dalley, R. Cardinal, T. Robbins (2004)
Prefrontal executive and cognitive functions in rodents: neural and neurochemical substratesNeuroscience & Biobehavioral Reviews, 28
R. Kyd, D. Bilkey (2003)
Prefrontal cortex lesions modify the spatial properties of hippocampal place cells.Cerebral cortex, 13 5
S. Carmichael, J. Price (1996)
Connectional networks within the orbital and medial prefrontal cortex of macaque monkeysJournal of Comparative Neurology, 371
G. Barker, Paul Banks, H. Scott, G. Ralph, K. Mitrophanous, Liang-Fong Wong, Z. Bashir, J. Uney, E. Warburton (2017)
Separate elements of episodic memory subserved by distinct hippocampal–prefrontal connectionsNature Neuroscience, 20
R. Cardinal, David Pennicott, C. Lakmali, Sugathapala, T. Robbins, B. Everitt (2001)
Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens CoreScience, 292
S. Shushruth, M. Mazurek, M. Shadlen (2017)
Comparison of Decision-Related Signals in Sensory and Motor Preparatory Responses of Neurons in Area LIPThe Journal of Neuroscience, 38
J. Ladenbauer, J. Ladenbauer, N. Külzow, Rebecca Boor, Elena Avramova, U. Grittner, A. Flöel (2017)
Promoting Sleep Oscillations and Their Functional Coupling by Transcranial Stimulation Enhances Memory Consolidation in Mild Cognitive ImpairmentThe Journal of Neuroscience, 37
C. Howard, Hao Li, Claire Geddes, Xin Jin (2017)
Dynamic Nigrostriatal Dopamine Biases Action SelectionNeuron, 93
P. Goldman-Rakic (1996)
The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive.Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 351 1346
Theresa Desrochers, Christopher Chatham, David Badre (2015)
The Necessity of Rostrolateral Prefrontal Cortex for Higher-Level Sequential BehaviorNeuron, 87
A. Genovesio, S. Tsujimoto, S. Wise (2012)
Encoding Goals but Not Abstract Magnitude in the Primate Prefrontal CortexNeuron, 74
Klaus Wunderlich, P. Dayan, R. Dolan (2012)
Mapping value based planning and extensively trained choice in the human brainNature neuroscience, 15
A. Mulder, R. Nordquist, Okyay Örgüt, C. Pennartz (2003)
Learning-related changes in response patterns of prefrontal neurons during instrumental conditioningBehavioural Brain Research, 146
J. Bruin, M. Feenstra, L. Broersen, Marieke Leeuwen, Claire Arens, Sanne Vries, R. Joosten (2000)
Role of the prefrontal cortex of the rat in learning and decision making: effects of transient inactivation.Progress in brain research, 126
K. Jimura, M. Chushak, T. Braver (2013)
Impulsivity and Self-Control during Intertemporal Decision Making Linked to the Neural Dynamics of Reward Value RepresentationThe Journal of Neuroscience, 33
C. Ranganath (2010)
A unified framework for the functional organization of the medial temporal lobes and the phenomenology of episodic memoryHippocampus, 20
M. Mölle, J. Born (2011)
Slow oscillations orchestrating fast oscillations and memory consolidation.Progress in brain research, 193
M. McDannald, Federica Lucantonio, K. Burke, Y. Niv, G. Schoenbaum (2011)
Ventral Striatum and Orbitofrontal Cortex Are Both Required for Model-Based, But Not Model-Free, Reinforcement LearningThe Journal of Neuroscience, 31
Yu Qin, B. McNaughton, W. Skaggs, C. Barnes (1997)
Memory reprocessing in corticocortical and hippocampocortical neuronal ensembles.Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 352 1360
J. Serences (2008)
Value-Based Modulations in Human Visual CortexNeuron, 60
S. Wise (2008)
Forward frontal fields: phylogeny and fundamental functionTrends in Neurosciences, 31
G. Schoenbaum, M. Roesch, T. Stalnaker, Yuji Takahashi (2009)
A new perspective on the role of the orbitofrontal cortex in adaptive behaviourNature Reviews Neuroscience, 10
A. Marcel (1983)
Conscious and unconscious perception: An approach to the relations between phenomenal experience and perceptual processesCognitive Psychology, 15
P. Goltstein, Emily Coffey, P. Roelfsema, C. Pennartz (2013)
In Vivo Two-Photon Ca2+ Imaging Reveals Selective Reward Effects on Stimulus-Specific Assemblies in Mouse Visual CortexThe Journal of Neuroscience, 33
R. Rescorla, A. Wagner (1972)
A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement
Colin McNamara, Álvaro Tejero-Cantero, S. Trouche, Natalia Campo-Urriza, D. Dupret (2014)
Dopaminergic neurons promote hippocampal reactivation and spatial memory persistenceNature neuroscience, 17
J. Tamminen, M. Ralph, P. Lewis (2013)
The Role of Sleep Spindles and Slow-Wave Activity in Integrating New Information in Semantic MemoryThe Journal of Neuroscience, 33
H. Eichenbaum, M. Sauvage, N. Fortin, Robert Komorowski, P. Lipton (2012)
Towards a functional organization of episodic memory in the medial temporal lobeNeuroscience & Biobehavioral Reviews, 36
H. Eichenbaum (2008)
Faculty Opinions recommendation of Internally generated cell assembly sequences in the rat hippocampus.
J. Knierim (2009)
Faculty Opinions recommendation of Selective suppression of hippocampal ripples impairs spatial memory.
Shigeyoshi Fujisawa, G. Buzsáki (2011)
A 4 Hz Oscillation Adaptively Synchronizes Prefrontal, VTA, and Hippocampal ActivitiesNeuron, 72
V. Ego-Stengel, M. Wilson (2009)
Disruption of ripple‐associated hippocampal activity during rest impairs spatial learning in the ratHippocampus, 20
H. Groenewegen, H. Uylings (2000)
The prefrontal cortex and the integration of sensory, limbic and autonomic information.Progress in brain research, 126
K. Samejima, Y. Ueda, K. Doya, M. Kimura (2005)
Representation of Action-Specific Reward Values in the StriatumScience, 310
G. Alexander, M. Crutcher, M. DeLong (1990)
Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, "prefrontal" and "limbic" functions.Progress in brain research, 85
J. Berke (2018)
What does dopamine mean?Nature Neuroscience, 21
J. Reynolds, B. Hyland, J. Wickens (2001)
A cellular mechanism of reward-related learningNature, 413
E. Rolls (2000)
The orbitofrontal cortex and reward.Cerebral cortex, 10 3
B. Balleine, J. O’Doherty (2010)
Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual ActionNeuropsychopharmacology, 35
How to cite this article: Rusu SI, Pennartz CMA. Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico-basal ganglia systems
J. Berke, Murat Okatan, Jennifer Skurski, H. Eichenbaum (2004)
Oscillatory Entrainment of Striatal Neurons in Freely Moving RatsNeuron, 43
L. Corbit, Sean Ostlund, B. Balleine (2002)
Sensitivity to Instrumental Contingency Degradation Is Mediated by the Entorhinal Cortex and Its Efferents via the Dorsal HippocampusThe Journal of Neuroscience, 22
R. Borchgrave, J. Rawlins, A. Dickinson, B. Balleine (2002)
Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in ratsExperimental Brain Research, 144
L. Corbit, B. Balleine (2003)
The role of prelimbic cortex in instrumental conditioningBehavioural Brain Research, 146
Y. Dongen, J. Deniau, C. Pennartz, Y. Graaf, Pieter Voorn, A. Thierry, H. Groenewegen (2005)
Anatomical evidence for direct connections between the shell and core subregions of the rat nucleus accumbensNeuroscience, 136
H. Eichenbaum (2014)
Time cells in the hippocampus: a new dimension for mapping memoriesNature Reviews Neuroscience, 15
S. Sesack, A. Grace (2010)
Cortico-Basal Ganglia Reward Network: MicrocircuitryNeuropsychopharmacology, 35
Y. Maurin, B. Banrezes, A. Menétrey, Philippe Mailly, J. Deniau (1999)
Three-dimensional distribution of nigrostriatal neurons in the rat: relation to the topography of striatonigral projectionsNeuroscience, 91
W. Schultz, P. Dayan, P. Montague (1997)
A Neural Substrate of Prediction and RewardScience, 275
Makoto Ito, K. Doya (2015)
Distinct Neural Representation in the Dorsolateral, Dorsomedial, and Ventral Parts of the Striatum during Fixed- and Free-Choice TasksThe Journal of Neuroscience, 35
E. Pastalkova, V. Itskov, A. Amarasingham, G. Buzsáki (2008)
Internally generated cell assembly sequences in the rat hippocampus, 321
R. Ito, T. W. Robbins, C. M. Pennartz, B. J. Everitt (2008)
Functional interaction between the hippocampus and nucleus accumbens shell is necessary for the acquisition of appetitive spatial context conditioning, 28
(2008)
Spontaneous changes of neocortical code for associative memory during consolidationScience (New York, N.Y.), 322
M. Roesch, Teghpal Singh, P. Brown, Sylvina Mullins, G. Schoenbaum (2009)
Ventral Striatal Neurons Encode the Value of the Chosen Action in Rats Deciding between Differently Delayed or Sized RewardsThe Journal of Neuroscience, 29
Seth Ramus, H. Eichenbaum (2000)
Neural Correlates of Olfactory Recognition Memory in the Rat Orbitofrontal CortexThe Journal of Neuroscience, 20
C. Pennartz, M. Wingerden, M. Vinck (2011)
Population coding and neural rhythmicity in the orbitofrontal cortexAnnals of the New York Academy of Sciences, 1239
C. Lansink, P. Goltstein, J. Lankelma, C. Pennartz (2010)
Fast‐spiking interneurons of the rat ventral striatum: temporal coordination of activity with principal cells and responsiveness to rewardEuropean Journal of Neuroscience, 32
William Allen, Isaac Kauvar, Michael Chen, Ethan Richman, Samuel Yang, Ken Chan, V. Gradinaru, Benjamin Deverman, L. Luo, K. Deisseroth (2017)
Global Representations of Goal-Directed Behavior in Distinct Cell Types of Mouse NeocortexNeuron, 94
B. Balleine, A. Dickinson (1998)
Goal-directed instrumental action: contingency and incentive learning and their cortical substratesNeuropharmacology, 37
J. Haddon, S. Killcross (2006)
Prefrontal Cortex Lesions Disrupt the Contextual Control of Response ConflictThe Journal of Neuroscience, 26
Yuji Takahashi, Angela Langdon, Y. Niv, G. Schoenbaum (2016)
Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral StriatumNeuron, 91
G. Handelmann, D. Olton (1981)
Spatial memory following damage to hippocampal CA3 pyramidal cells with kainic acid: impairment and recovery with preoperative trainingBrain Research, 217
Jimmie Gmaz, J. Carmichael, Matthijs Meer (2018)
Persistent coding of outcome-predictive cue features in the rat nucleus accumbenseLife, 7
M. Wingerden, M. Vinck, V. Tijms, Irene Ferreira, A. Jonker, C. Pennartz (2012)
NMDA Receptors Control Cue-Outcome Selectivity and Plasticity of Orbitofrontal Firing Patterns during Associative Stimulus-Reward LearningNeuron, 76
C. Lansink, P. Goltstein, J. Lankelma, R. Joosten, B. McNaughton, C. Pennartz (2008)
Preferential Reactivation of Motivationally Relevant Information in the Ventral StriatumThe Journal of Neuroscience, 28
Kyle Smith, A. Graybiel (2013)
A Dual Operator View of Habitual Behavior Reflecting Cortical and Striatal DynamicsNeuron, 79
G. Girardeau, K. Benchenane, S. I. Wiener, G. Buzsáki, M. B. Zugaro (2009)
Selective suppression of hippocampal ripples impairs spatial memory, 12
Sean Ostlund, B. Balleine (2007)
Orbitofrontal Cortex Mediates Outcome Encoding in Pavlovian But Not Instrumental ConditioningThe Journal of Neuroscience, 27
C. Pennartz (1997)
Reinforcement learning by Hebbian synapses with adaptive thresholdsNeuroscience, 81
J. Houk, Joel Davis, D. Beiser (1994)
A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement
T. Hosokawa, S. Kennerley, J. Sloan, J. Wallis (2013)
Single-Neuron Mechanisms Underlying Cost-Benefit Analysis in Frontal CortexThe Journal of Neuroscience, 33
M. McDannald, Yuji Takahashi, Nina Lopatina, B. Pietras, Josh Jones, G. Schoenbaum (2012)
Model‐based learning and the contribution of the orbitofrontal cortex to the model‐free worldEuropean Journal of Neuroscience, 35
T. Stalnaker, Gwendolyn Calhoon, Masaaki Ogawa, M. Roesch, Geoffrey Schoenbaum (2010)
Neural Correlates of Stimulus–Response and Response–Outcome Associations in Dorsolateral Versus Dorsomedial StriatumFrontiers in Integrative Neuroscience, 4
Brad Pfeiffer, David Foster (2013)
Hippocampal place cell sequences depict future paths to remembered goalsNature, 497
V. Denenberg, Douglas Kim, R. Palmiter (2004)
The role of dopamine in learning, memory, and performance of a water escape taskBehavioural Brain Research, 148
M. Botvinick, Y. Niv, A. Barto (2009)
Hierarchically organized behavior and its neural foundations: A reinforcement learning perspectiveCognition, 113
P. O’Neill, J. Gordon, Torfi Sigurdsson (2013)
Theta Oscillations in the Medial Prefrontal Cortex Are Modulated by Spatial Working Memory and Synchronize with the Hippocampus through Its Ventral SubregionThe Journal of Neuroscience, 33
C. Latchoumane, H. Ngo, J. Born, Hee-Sup Shin (2017)
Thalamic Spindles Promote Memory Formation during Sleep through Triple Phase-Locking of Cortical, Thalamic, and Hippocampal RhythmsNeuron, 95
Ethan Bromberg-Martin, Masayuki Matsumoto, O. Hikosaka (2010)
Dopamine in Motivational Control: Rewarding, Aversive, and AlertingNeuron, 68
J. Valdés, B. McNaughton, J. Fellous (2015)
Offline reactivation of experience-dependent neuronal firing patterns in the rat ventral tegmental area.Journal of neurophysiology, 114 2
S. Gomperts, F. Kloosterman, M. Wilson (2015)
VTA neurons coordinate with the hippocampal reactivation of spatial experienceeLife, 4
G. Mogenson, Douglas Jones, C. Yim (1980)
From motivation to action: Functional interface between the limbic system and the motor systemProgress in Neurobiology, 14
Matthijs Meer, A. Redish (2011)
Theta Phase Precession in Rat Ventral Striatum Links Place and Reward InformationThe Journal of Neuroscience, 31
K. Benchenane, A. Peyrache, M. Khamassi, P. Tierney, Y. Gioanni, F. Battaglia, S. Wiener (2010)
Coherent Theta Oscillations and Reorganization of Spike Timing in the Hippocampal- Prefrontal Network upon LearningNeuron, 66
Joshua Jones, Guillem Esber, M. McDannald, A. Gruber, Alex Hernandez, Aaron Mirenzi, G. Schoenbaum (2012)
Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached ValuesScience, 338
Mitsuko Watabe-Uchida, Lisa Zhu, Sachie Ogawa, Archana Vamanrao, N. Uchida (2012)
Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine NeuronsNeuron, 74
M. Wingerden, M. Vinck, J. Lankelma, C. Pennartz (2010)
Theta-Band Phase Locking of Orbitofrontal Neurons during Reward ExpectancyThe Journal of Neuroscience, 30
T. Thesen, Chunmao Wang, O. Devinsky, R. Kuzniecky, W. Doyle, J. Madsen, E. Bromfield, L. Erőss, P. Halász, G. Karmos, Richárd Csercsa, L. Wittner, I. Ulbert (2009)
Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral ConditioningScience, 324
G. Schoenbaum, B. Setlow, Michael Saddoris, M. Gallagher (2003)
Encoding Predicted Outcome and Acquired Value in Orbitofrontal Cortex during Cue Sampling Depends upon Input from Basolateral AmygdalaNeuron, 39
W. Schultz, A. Ruffieux, P. Aebischer (1983)
The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activationExperimental Brain Research, 51
Benjamin Kraus, Robert Robinson, J. White, H. Eichenbaum, M. Hasselmo (2013)
Hippocampal “Time Cells”: Time versus Path IntegrationNeuron, 78
T. Allen, Daniel Salz, S. McKenzie, N. Fortin (2016)
Nonspatial Sequence Coding in CA1 NeuronsThe Journal of Neuroscience, 36
S. Gunawardena, W. Rivera, Epstein, C Heathcock, B. Finkelstein, T. Aoki, C. Poulter, D. Jarstfer, C. Zhang, J. Poulter, Am, Chem, B Blagg, M. Jarstfer, D. Rogers, B Bierl-Leonhardt, D. Moreno, M. Schwarz, J. Fargerlund, J. Plimmer, L Tarshis, M. Yan, J. Sacchettini, A Hemmerlin, S. Rivera, H. Erickson, H Erickson, L Altman, R. Kowerski, D. Laungani, J. Am, D. Tse, R. Langston, M. Kakeyama, I. Bethus, P. Spooner, E. Wood, M. Witter, R. Morris
References and Notes Supporting Online Material Materials and Methods Figs. S1 to S5 Tables S1 to S3 Electron Impact (ei) Mass Spectra Chemical Ionization (ci) Mass Spectra References Schemas and Memory Consolidation Research Articles Research Articles Research Articles Research Articles
David Redish (2016)
Vicarious trial and errorNature Reviews Neuroscience, 17
J. Hall, J. Parkinson, Thomas Connor, A. Dickinson, B. Everitt (2001)
Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviourEuropean Journal of Neuroscience, 13
L. Corbit, B. Balleine (2000)
The Role of the Hippocampus in Instrumental ConditioningThe Journal of Neuroscience, 20
Megan Shipman, Sydney Trask, M. Bouton, John Green (2018)
Inactivation of prelimbic and infralimbic cortex respectively affects minimally-trained and extensively-trained goal-directed actionsNeurobiology of Learning and Memory, 155
T. Dugladze, D. Schmitz, M. Whittington, I. Vida, T. Gloveli (2012)
Awake Hippocampal Sharp-Wave Ripples Support Spatial MemoryScience, 336
Laetitia Lalla, Pavel Orozco, M. Jurado-Parras, A. Brovelli, David Robbe (2017)
Local or Not Local: Investigating the Nature of Striatal Theta Oscillations in Behaving RatseNeuro, 4
W. Shen, M. Flajolet, P. Greengard, D. Surmeier (2008)
Dichotomous Dopaminergic Control of Striatal Synaptic PlasticityScience, 321
Gideon Rothschild, Elad Eban, L. Frank (2016)
A cortical–hippocampal–cortical loop of information processing during memory consolidationNature Neuroscience, 20
Stephen Cowen, Glen Davis, D. Nitz (2012)
Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequences.Journal of neurophysiology, 107 9
A. Bastos, W. Usrey, Rick Adams, G. Mangun, P. Fries, Karl Friston (2012)
Canonical Microcircuits for Predictive CodingNeuron, 76
R. Ito, T. Robbins, C. Pennartz, B. Everitt (2008)
Version unknown SOURCE ( OR PART OF THE FOLLOWING SOURCE ) : Type article Title Functional interaction between the hippocampus and nucleus accumbens shell is necessary for the acquisition of appetitive spatial context conditioning
P. Rudebeck, M. Walton, Angharad Smyth, D. Bannerman, M. Rushworth (2006)
Separate neural pathways process different decision costsNature Neuroscience, 9
Erin Rich, M. Shapiro (2009)
Rat Prefrontal Cortical Neurons Selectively Code Strategy SwitchesThe Journal of Neuroscience, 29
Yuji Takahashi, Hannah Batchelor, Bing Liu, Akash Khanna, M. Morales, G. Schoenbaum (2017)
Dopamine Neurons Respond to Errors in the Prediction of Sensory Features of Expected RewardsNeuron, 95
S. Geisler, C. Derst, R. Veh, D. Zahm (2007)
Glutamatergic Afferents of the Ventral Tegmental Area in the RatThe Journal of Neuroscience, 27
A. Barbey, F. Krueger, J. Grafman (2009)
Structured event complexes in the medial prefrontal cortex support counterfactual representations for future planningPhilosophical Transactions of the Royal Society B: Biological Sciences, 364
A. Peyrache, M. Khamassi, K. Benchenane, S. Wiener, F. Battaglia (2009)
Replay of rule-learning related neural patterns in the prefrontal cortex during sleepNature Neuroscience, 12
T. Robbins, B. Everitt (1996)
Neurobehavioural mechanisms of reward and motivationCurrent Opinion in Neurobiology, 6
J. Fuster (2001)
The Prefrontal Cortex—An Update Time Is of the EssenceNeuron, 30
B. Milner, L. Squire, E. Kandel (1998)
Cognitive Neuroscience and the Study of MemoryNeuron, 20
P. Hitchcott, J. Quinn, JaneR Taylor (2007)
Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine.Cerebral cortex, 17 12
David Foster, M. Wilson (2006)
Reverse replay of behavioural sequences in hippocampal place cells during the awake stateNature, 440
P. Montague (2005)
Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks
Jeroen Bos, M. Vinck, Laura Mourik-Donga, Jadin Jackson, M. Witter, C. Pennartz (2017)
Perirhinal firing patterns are sustained across large spatial segments of the task environmentNature Communications, 8
R. Kesner, M. Hunsaker, Matthew Warthen (2008)
The CA3 subregion of the hippocampus is critical for episodic memory processing by means of relational encoding in rats.Behavioral neuroscience, 122 6
H. Eichenbaum (2013)
Memory on timeTrends in Cognitive Sciences, 17
V. Pawlak, J. Kerr (2008)
Dopamine Receptor Activation Is Required for Corticostriatal Spike-Timing-Dependent PlasticityThe Journal of Neuroscience, 28
I. Stoianov, C. Pennartz, C. Lansink, G. Pezzulo (2018)
Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysisPLoS Computational Biology, 14
H. Groenewegen, H. Berendse, J. Wolters, A. Lohman (1990)
The anatomical relationship of the prefrontal cortex with the striatopallidal system, the thalamus and the amygdala: evidence for a parallel organization.Progress in brain research, 85
A. Barto (1994)
Reinforcement learning controlCurrent Opinion in Neurobiology, 4
N. Daw, P. Dayan (2014)
The algorithmic anatomy of model-based evaluationPhilosophical Transactions of the Royal Society B: Biological Sciences, 369
M. Darvas, A. Wunsch, J. Gibbs, R. Palmiter (2014)
Dopamine dependency for acquisition and performance of Pavlovian conditioned responseProceedings of the National Academy of Sciences, 111
S. Taverna, B. Canciani, C. Pennartz (2007)
Membrane properties and synaptic connectivity of fast-spiking interneurons in rat ventral striatumBrain Research, 1152
L. Marshall, H. Helgadóttir, M. Mölle, J. Born (2006)
Boosting slow oscillations during sleep potentiates memoryNature, 444
Hyoung Kim, O. Hikosaka (2013)
Distinct Basal Ganglia Circuits Controlling Behaviors Guided by Flexible and Stable ValuesNeuron, 79
V. Crunelli, S. Hughes (2010)
The slow (<1 Hz) rhythm of non-REM sleep: a dialogue between three cardinal oscillatorsNature Neuroscience, 13
P. Verschure, C. Pennartz, G. Pezzulo (2014)
The why, what, where, when and how of goal-directed choice: neuronal and computational principlesPhilosophical Transactions of the Royal Society B: Biological Sciences, 369
P. Holland (2004)
Relations between Pavlovian-instrumental transfer and reinforcer devaluation.Journal of experimental psychology. Animal behavior processes, 30 2
G. Hart, Laura Bradfield, B. Balleine (2018)
Prefrontal Corticostriatal Disconnection Blocks the Acquisition of Goal-Directed ActionThe Journal of Neuroscience, 38
C. Gremel, R. Costa (2013)
Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actionsNature communications, 4
A. Sirota, J. Csicsvari, D. Buhl, G. Buzsáki (2003)
Communication between neocortex and hippocampus during sleep in rodentsProceedings of the National Academy of Sciences of the United States of America, 100
T. Kalenscher, C. Pennartz (2008)
Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-makingProgress in Neurobiology, 84
C. Pennartz, J. Berke, A. Graybiel, R. Ito, C. Lansink, Matthijs Meer, A. Redish, Kyle Smith, P. Voorn (2009)
Corticostriatal Interactions during Learning, Memory Processing, and Decision MakingThe Journal of Neuroscience, 29
E. Coutureau, S. Killcross (2003)
Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained ratsBehavioural Brain Research, 146
Mitsuko Watabe-Uchida, N. Eshel, N. Uchida (2017)
Neural Circuitry of Reward Prediction Error.Annual review of neuroscience, 40
S. Bao, Vincent Chan, M. Merzenich (2001)
Cortical remodelling induced by activity of ventral tegmental dopamine neuronsNature, 412
D. Euston, A. Gruber, B. McNaughton (2012)
The Role of Medial Prefrontal Cortex in Memory and Decision MakingNeuron, 76
E. Miller, J. Cohen (2001)
An integrative theory of prefrontal cortex function.Annual review of neuroscience, 24
M. Leinweber, D. Ward, Jan Sobczak, Alexander Attinger, Georg Keller (2017)
A Sensorimotor Circuit in Mouse Cortex for Visual Flow PredictionsNeuron, 95
N. Fortin, Kara Agster, H. Eichenbaum (2002)
Critical role of the hippocampus in memory for sequences of eventsNature Neuroscience, 5
Dennis Burke, H. Rotstein, V. Alvarez (2017)
Striatal Local Circuitry: A New Framework for Lateral InhibitionNeuron, 96
Lise Johnson, T. Blakely, D. Hermes, S. Hakimian, N. Ramsey, J. Ojemann (2012)
Sleep spindles are locally modulated by training on a brain–computer interfaceProceedings of the National Academy of Sciences, 109
G. Buzsáki (1986)
Hippocampal sharp waves: Their origin and significanceBrain Research, 398
S. Taverna, Yvette Dongen, H. Groenewegen, C. Pennartz (2004)
Direct physiological evidence for synaptic connectivity between medium-sized spiny neurons in rat nucleus accumbens in situ.Journal of neurophysiology, 91 3
A. Belin-Rauscent, B. Everitt, D. Belin (2012)
Intrastriatal Shifts Mediate the Transition from Drug-Seeking Actions to HabitsBiological Psychiatry, 72
P. Calabresi, B. Picconi, A. Tozzi, M. Filippo (2007)
Dopamine-mediated regulation of corticostriatal synaptic plasticityTrends in Neurosciences, 30
Walter Hoover, R. Vertes (2007)
Anatomical analysis of afferent projections to the medial prefrontal cortex in the ratBrain Structure and Function, 212
Niels Hansen, D. Manahan‐Vaughan (2012)
Dopamine D1/D5 Receptors Mediate Informational Saliency that Promotes Persistent Hippocampal Long-Term PlasticityCerebral Cortex (New York, NY), 24
T. Kitamura, Sachie Ogawa, Dheeraj Roy, T. Okuyama, Mark Morrissey, Lillian Smith, R. Redondo, S. Tonegawa (2017)
Engrams and circuits crucial for systems consolidation of a memoryScience, 356
Karl Friston (2010)
The free-energy principle: a unified brain theory?Nature Reviews Neuroscience, 11
G. Buzsáki (1989)
Two-stage model of memory trace formation: A role for “noisy” brain statesNeuroscience, 31
R. O’Reilly, M. Frank (2006)
Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal GangliaNeural Computation, 18
Nisha Cooch, T. Stalnaker, H. Wied, Sheena Chaudhary, M. McDannald, T. Liu, G. Schoenbaum (2015)
Orbitofrontal lesions eliminate signaling of biological significance in cue-responsive ventral striatal neuronsNature communications, 6
Anita Lüthi (2014)
Sleep SpindlesThe Neuroscientist, 20
H. Groenewegen, H. Berendse, S. Haber (1993)
Organization of the output of the ventral striatopallidal system in the rat: Ventral pallidal efferentsNeuroscience, 57
M. Moscovitch, R. Rosenbaum, A. Gilboa, D. Addis, R. Westmacott, C. Grady, M. McAndrews, B. Levine, S. Black, G. Winocur, L. Nadel (2005)
Functional neuroanatomy of remote episodic, semantic and spatial memory: a unified account based on multiple trace theoryJournal of Anatomy, 207
Simon Fisher, Paul Robertson, M. Black, P. Redgrave, Mark Sagar, W. Abraham, J. Reynolds (2017)
Reinforcement determines the timing dependence of corticostriatal synaptic plasticity in vivoNature Communications, 8
J. Salamone, M. Cousins, Sherri Bucher (1994)
Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedureBehavioural Brain Research, 65
T. Lee, D. Mumford (2003)
Hierarchical Bayesian inference in the visual cortex.Journal of the Optical Society of America. A, Optics, image science, and vision, 20 7
T. Vickery, M. Chun, Daeyeol Lee (2011)
Ubiquity and Specificity of Reinforcement Signals throughout the Human BrainNeuron, 72
Mark Thomas, R. Malenka, A. Bonci (2000)
Modulation of Long-Term Depression by Dopamine in the Mesolimbic SystemThe Journal of Neuroscience, 20
A. Flaherty, A. Graybiel (1995)
Motor and somatosensory corticostriatal projection magnifications in the squirrel monkey.Journal of neurophysiology, 74 6
Shauna Parkes, P. Ravassard, J. Cerpa, M. Wolff, G. Ferreira, E. Coutureau (2018)
Insular and Ventrolateral Orbitofrontal Cortices Differentially Contribute to Goal-Directed Behavior in RodentsCerebral Cortex, 28
J. Gläscher, N. Daw, P. Dayan, J. O’Doherty (2010)
States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement LearningNeuron, 66
T. Preuss (1995)
Do Rats Have Prefrontal Cortex? The Rose-Woolsey-Akert Program ReconsideredJournal of Cognitive Neuroscience, 7
(2010b)
Learning‐associated gamma‐band phase‐locking of action‐outcome selective neurons in orbitofrontal cortexThe Journal of Neuroscience, 30
D. Euston, M. Tatsuno, B. McNaughton (2007)
Fast-Forward Playback of Recent Memory Sequences in Prefrontal Cortex During SleepScience, 318
David Bett, Elizabeth Allison, Lauren Murdoch, Karola Kaefer, E. Wood, P. Dudchenko (2012)
The neural substrates of deliberative decision making: contrasting effects of hippocampus lesions on performance and vicarious trial-and-error behavior in a spatial memory task and a visual discrimination taskFrontiers in Behavioral Neuroscience, 6
K. Lashley (1951)
The problem of serial order in behavior
J. Mirenowicz, W. Schultz (1994)
Importance of unpredictability for reward responses in primate dopamine neurons.Journal of neurophysiology, 72 2
W. Schultz (1998)
Predictive reward signal of dopamine neurons.Journal of neurophysiology, 80 1
Satoshi Nonomura, Kayo Nishizawa, Y. Sakai, Y. Kawaguchi, S. Kato, M. Uchigashima, Masahiko Watanabe, K. Yamanaka, Kazuki Enomoto, S. Chiken, H. Sano, S. Soma, Junichi Yoshida, K. Samejima, Masaaki Ogawa, Kazuto Kobayashi, A. Nambu, Y. Isomura, M. Kimura (2018)
Monitoring and Updating of Action Selection for Goal-Directed Behavior through the Striatal Direct and Indirect PathwaysNeuron, 99
A. Sadek, P. Magill, J. Bolam (2007)
A Single-Cell Analysis of Intrinsic Connectivity in the Rat Globus PallidusThe Journal of Neuroscience, 27
E. Duuren, G. Plasse, J. Lankelma, R. Joosten, M. Feenstra, C. Pennartz (2009)
Single-Cell and Population Coding of Expected Reward Probability in the Orbitofrontal Cortex of the RatThe Journal of Neuroscience, 29
D. Aronov, Rhino Nevers, D. Tank (2017)
Mapping of a non-spatial dimension by the hippocampal/entorhinal circuitNature, 543
G. Miller, E. Galanter, K. Pribram (1960)
Plans and the structure of behavior
Hiroshi Ito, Sheng-Jia Zhang, M. Witter, E. Moser, M. Moser (2015)
A prefrontal–thalamo–hippocampal circuit for goal-directed spatial navigationNature, 522
M. DeLong, M. Crutcher, A. Georgopoulos (1983)
Relations between movement and single cell discharge in the substantia nigra of the behaving monkey, 3
C. Lansink, P. Goltstein, J. Lankelma, B. McNaughton, C. Pennartz (2009)
Hippocampus Leads Ventral Striatum in Replay of Place-Reward InformationPLoS Biology, 7
K. Burke, Yuji Takahashi, Jessica Correll, P. Brown, G. Schoenbaum, G. Schoenbaum (2009)
Orbitofrontal inactivation impairs reversal of Pavlovian learning by interfering with ‘disinhibition’ of responding for previously unrewarded cuesEuropean Journal of Neuroscience, 30
Kaori Takehara-Nishiuchi, B. McNaughton (2008)
Supporting Online Material for Spontaneous Changes of Neocortical Code for Associative Memory During Consolidation
C. Pennartz (2015)
The Brain's Representational Power: On Consciousness and the Integration of Modalities
Vivian Valentin, A. Dickinson, J. O’Doherty (2007)
Determining the Neural Substrates of Goal-Directed Learning in the Human BrainThe Journal of Neuroscience, 27
R. Morris, Elizabeth Anderson, G. Lynch, M. Baudry (1986)
Selective impairment of learning and blockade of long-term potentiation by an N-methyl-D-aspartate receptor antagonist, AP5Nature, 319
E. Koechlin, Alexandre Hyafil (2007)
Anterior Prefrontal Function and the Limits of Human Decision-MakingScience, 318
P. Frankland, B. Bontempi (2005)
The organization of recent and remote memoriesNature Reviews Neuroscience, 6
H. Critchley, E. Rolls (1996)
Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex.Journal of neurophysiology, 75 4
Marina Weber (2016)
Elements Of Episodic Memory
C. Weiss, J. Disterhoft (2015)
The impact of hippocampal lesions on trace-eyeblink conditioning and forebrain-cerebellar interactions.Behavioral neuroscience, 129 4
C. Pennartz, R. Ito, R. Ito, P. Verschure, F. Battaglia, T. Robbins (2011)
The hippocampal–striatal axis in learning, prediction and goal-directed behaviorTrends in Neurosciences, 34
C. MacDonald, K. Lepage, U. Eden, H. Eichenbaum (2011)
Hippocampal “Time Cells” Bridge the Gap in Memory for Discontiguous EventsNeuron, 71
Y. Nir, R. Staba, T. Andrillon, V. Vyazovskiy, C. Cirelli, I. Fried, G. Tononi (2011)
Regional Slow Waves and Spindles in Human SleepNeuron, 70
H. Eichenbaum, P. Dudchenko, E. Wood, M. Shapiro, H. Tanila (1999)
The Hippocampus, Memory, and Place Cells Is It Spatial Memory or a Memory Space?Neuron, 23
H. Groenewegen, E. Zee, A. Kortschot, M. Witter (1987)
Organization of the projections from the subiculum to the ventral striatum in the rat. A study using anterograde transport of Phaseolus vulgaris leucoagglutininNeuroscience, 23
Nicolas Maingret, Gabrielle Girardeau, Ralitsa Todorova, Marie Goutierre, Michaël Zugaro (2016)
Hippocampo-cortical coupling mediates memory consolidation during sleepNature Neuroscience, 19
M. Eckardt (1980)
The Hippocampus as a Cognitive MapJournal of Nervous and Mental Disease, 168
W. Schultz, Wiliam Stauffer, Armin Lak (2017)
The phasic dopamine signal maturing: from reward via behavioural activation to formal economic utilityCurrent Opinion in Neurobiology, 43
W. Schultz (2016)
Dopamine reward prediction-error signalling: a two-component responseNature Reviews Neuroscience, 17
C. Azuar, Pablo Reyes, A. Slachevsky, Emmanuelle Volle, Emmanuelle Volle, S. Kinkingnéhun, S. Kinkingnéhun, F. Kouneiher, E. Bravo, Bruno Dubois, Bruno Dubois, E. Koechlin, Richard Levy, Richard Levy (2014)
Testing the model of caudo-rostral organization of cognitive control in the human with frontal lesionsNeuroImage, 84
M. Wilson, B. McNaughton (1993)
Dynamics of the hippocampal ensemble code for space.Science, 261 5124
M. Walton, P. Croxson, T. Behrens, S. Kennerley, M. Rushworth (2007)
Adaptive decision making and value in the anterior cingulate cortexNeuroImage, 36
Feng-Kuei Chiang, J. Wallis (2018)
Neuronal Encoding in Prefrontal Cortex during Hierarchical Reinforcement LearningJournal of Cognitive Neuroscience, 30
C. Lansink, Guido Meijer, J. Lankelma, M. Vinck, Jadin Jackson, C. Pennartz (2016)
Reward Expectancy Strengthens CA1 Theta and Beta Band Synchronization and Hippocampal-Ventral Striatal CouplingThe Journal of Neuroscience, 36
Mathew Wilson, B. McNaughton (1994)
Reactivation of hippocampal ensemble memories during sleep.Science, 265 5172
C. Thorn, H. Atallah, Mark Howe, A. Graybiel (2010)
Differential Dynamics of Activity Changes in Dorsolateral and Dorsomedial Striatal Loops during LearningNeuron, 66
Matthijs Meer, A. Redish (2009)
Covert Expectation-of-Reward in Rat Ventral Striatum at Decision PointsFrontiers in Integrative Neuroscience, 3
Andrew Wikenheiser, Yasmin Marrero-Garcia, G. Schoenbaum (2017)
Suppression of Ventral Hippocampal Output Impairs Integrated Orbitofrontal Encoding of Task StructureNeuron, 95
Fabien Naneix, A. Marchand, G. Scala, J. Pape, E. Coutureau (2009)
A Role for Medial Prefrontal Dopaminergic Innervation in Instrumental ConditioningThe Journal of Neuroscience, 29
W. Meck, R. Church, D. Olton (1984)
Hippocampus, time, and memory.Behavioral neuroscience, 127 5
T. Teyler, P. Discenna (1986)
The hippocampal memory indexing theory.Behavioral neuroscience, 100 2
C. Lansink, Jadin Jackson, J. Lankelma, R. Ito, T. Robbins, B. Everitt, C. Pennartz (2012)
Reward Cues in Space: Commonalities and Differences in Neural Coding by Hippocampal and Ventral Striatal EnsemblesThe Journal of Neuroscience, 32
Rebecca Nordquist, Rebecca Nordquist, Pieter Voorn, J. Malsen, R. Joosten, C. Pennartz, L. Vanderschuren (2007)
Augmented reinforcer value and accelerated habit formation after repeated amphetamine treatmentEuropean Neuropsychopharmacology, 17
K. Berridge (2007)
The debate over dopamine’s role in reward: the case for incentive saliencePsychopharmacology, 191
D. Marr (1971)
Simple memory: a theory for archicortex.Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 262 841
S. Jackson, N. Horst, Andrew Pears, T. Robbins, A. Roberts (2016)
Role of the Perigenual Anterior Cingulate and Orbitofrontal Cortex in Contingency Learning in the MarmosetCerebral Cortex (New York, NY), 26
W. Scoville, B. Milner (1957)
LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONSJournal of Neurology, Neurosurgery & Psychiatry, 20
N. Daw, Y. Niv, P. Dayan (2005)
Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral controlNature Neuroscience, 8
R. Gregory (1980)
Perceptions as hypotheses.Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 290 1038
F. Battaglia, C. Pennartz
Source (or Part of the following Source): Type Article Title the Construction of Semantic Memory: Grammar-based Representations Learned from Relational Episodic Information. Author(s)
G. Winocur, M. Moscovitch, B. Bontempi (2010)
Memory formation and long-term retention in humans and animals: Convergence towards a transformation account of hippocampal–neocortical interactionsNeuropsychologia, 48
M. Massimini, R. Huber, F. Ferrarelli, Sean Hill, G. Tononi (2004)
The Sleep Slow Oscillation as a Traveling WaveThe Journal of Neuroscience, 24
B. Nolan (2008)
Boosting slow oscillations during sleep potentiates memoryYearbook of Neurology and Neurosurgery, 2008
William Menegas, J. Bergan, Sachie Ogawa, Yoh Isogai, Kannan Venkataraju, P. Osten, N. Uchida, Mitsuko Watabe-Uchida (2015)
Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclasseLife, 4
Paul Gabbott, T. Warner, Jacki Brown, Philip Salway, Thomas Gabbott, S. Busby (2012)
Amygdala afferents monosynaptically innervate corticospinal neurons in rat medial prefrontal cortexJournal of Comparative Neurology, 520
E. Duuren, J. Lankelma, C. Pennartz (2008)
Population Coding of Reward Magnitude in the Orbitofrontal Cortex of the RatThe Journal of Neuroscience, 28
D. Ji, M. Wilson (2007)
Coordinated memory replay in the visual cortex and hippocampus during sleepNature Neuroscience, 10
F. Battaglia, K. Benchenane, A. Sirota, C. Pennartz, S. Wiener (2011)
The hippocampus: hub of brain network communication for memoryTrends in Cognitive Sciences, 15
This article aims to provide a synthesis on the question how brain structures cooperate to accomplish hierarchically organized behaviors, characterized by low‐level, habitual routines nested in larger sequences of planned, goal‐directed behavior. The functioning of a connected set of brain structures—prefrontal cortex, hippocampus, striatum, and dopaminergic mesencephalon—is reviewed in relation to two important distinctions: (a) goal‐directed as opposed to habitual behavior and (b) model‐based and model‐free learning. Recent evidence indicates that the orbitomedial prefrontal cortices not only subserve goal‐directed behavior and model‐based learning, but also code the “landscape” (task space) of behaviorally relevant variables. While the hippocampus stands out for its role in coding and memorizing world state representations, it is argued to function in model‐based learning but is not required for coding of action–outcome contingencies, illustrating that goal‐directed behavior is not congruent with model‐based learning. While the dorsolateral and dorsomedial striatum largely conform to the dichotomy between habitual versus goal‐directed behavior, ventral striatal functions go beyond this distinction. Next, we contextualize findings on coding of reward‐prediction errors by ventral tegmental dopamine neurons to suggest a broader role of mesencephalic dopamine cells, viz. in behavioral reactivity and signaling unexpected sensory changes. We hypothesize that goal‐directed behavior is hierarchically organized in interconnected cortico‐basal ganglia loops, where a limbic‐affective prefrontal‐ventral striatal loop controls action selection in a dorsomedial prefrontal–striatal loop, which in turn regulates activity in sensorimotor‐dorsolateral striatal circuits. This structure for behavioral organization requires alignment with mechanisms for memory formation and consolidation. We propose that frontal corticothalamic circuits form a high‐level loop for memory processing that initiates and temporally organizes nested activities in lower‐level loops, including the hippocampus and the ripple‐associated replay it generates. The evidence on hierarchically organized behavior converges with that on consolidation mechanisms in suggesting a frontal‐to‐caudal directionality in processing control.
Hippocampus – Wiley
Published: Jan 1, 2020
Keywords: ; ; ; ; ;
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.