ArticlePublisher preview available

Goal-directed actions transiently depend on dorsal hippocampus

Authors:
Laura A Bradfield at University of Technology Sydney
Laura A Bradfield
Beatrice K Leung at UNSW Sydney
Beatrice K Leung
Susan Boldt
Susan Boldt
Susan Boldt
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Sophia Liang at UNSW Sydney
Sophia Liang
Show all 5 authorsHide
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

The role of the hippocampus in goal-directed action is currently unclear; studies investigating this issue have produced contradictory results. Here we reconcile these contradictions by demonstrating that, in rats, goal-directed action relies on the dorsal hippocampus, but only transiently, immediately after initial acquisition. Furthermore, we found that goal-directed action also depends transiently on physical context, suggesting a psychological basis for the hippocampal regulation of goal-directed action control.
Inactivation of the dorsal hippocampus transiently impaired outcome devaluation performance when testing was immediate, but not after additional training or a delay a, Design of Experiments 1a and 1b. b, Mean lever presses during the test in Experiment 1a (n = 19 rats), c, Mean lever presses during the test in Experiment 1b (n = 16 rats), d, Design of Experiment 2. e, Left: micrographs are representative of dorsal hippocampal neurons transfected with the DREADD virus AAV8-hSyn-hM4D(Gi)-mCherry (M4; red) in CA1 (n = 19 rats showed similar infection in Experiment 2) and lateral CA2 (n = 16 rats showed similar infection in Experiment 2) regions. Right: magnified views of the white boxed areas. f, Mean lever presses during the initial test (days 3–4; n = 35 rats). g, Mean presses during the extended test (days 9–10; n = 35 rats). h, Design of Experiments 3a and 3b. i, Mean lever presses during the test in Experiment 3a (n = 38 rats). j, Mean lever presses during the test in Experiment 3b (n = 37 rats). Data are shown as individual dot plots and the mean ± s.e.m. A, action; O, outcome.
Inactivation of the dorsal hippocampus transiently impaired outcome devaluation performance when testing was immediate, but not after additional training or a delay a, Design of Experiments 1a and 1b. b, Mean lever presses during the test in Experiment 1a (n = 19 rats), c, Mean lever presses during the test in Experiment 1b (n = 16 rats), d, Design of Experiment 2. e, Left: micrographs are representative of dorsal hippocampal neurons transfected with the DREADD virus AAV8-hSyn-hM4D(Gi)-mCherry (M4; red) in CA1 (n = 19 rats showed similar infection in Experiment 2) and lateral CA2 (n = 16 rats showed similar infection in Experiment 2) regions. Right: magnified views of the white boxed areas. f, Mean lever presses during the initial test (days 3–4; n = 35 rats). g, Mean presses during the extended test (days 9–10; n = 35 rats). h, Design of Experiments 3a and 3b. i, Mean lever presses during the test in Experiment 3a (n = 38 rats). j, Mean lever presses during the test in Experiment 3b (n = 37 rats). Data are shown as individual dot plots and the mean ± s.e.m. A, action; O, outcome.
… 
Outcome devaluation performance is impaired by a context switch immediately after limited training, but not after additional training or a delay a, Design of Experiments 4a and 4b. b, Mean lever presses during the test for rats that had limited training (Experiment 4a; n = 25 rats). c, Mean lever presses during the test for rats that had extended training (Experiment 4b; n = 21 rats). d, Design of Experiment 5. e, Mean presses during the devaluation test (n = 26 rats). Data are shown as individual dot plots and the mean ± s.e.m. A, action; O, outcome.
Outcome devaluation performance is impaired by a context switch immediately after limited training, but not after additional training or a delay a, Design of Experiments 4a and 4b. b, Mean lever presses during the test for rats that had limited training (Experiment 4a; n = 25 rats). c, Mean lever presses during the test for rats that had extended training (Experiment 4b; n = 21 rats). d, Design of Experiment 5. e, Mean presses during the devaluation test (n = 26 rats). Data are shown as individual dot plots and the mean ± s.e.m. A, action; O, outcome.
… 
Figures - available from: Nature Neuroscience
This content is subject to copyright. Terms and conditions apply.
A preview of this full-text is provided by Springer Nature.
Learn more
Content available from Nature Neuroscience
This content is subject to copyright. Terms and conditions apply.
Brief CommuniCation
https://doi.org/10.1038/s41593-020-0693-8
1Centre for Neuroscience and Regenerative Medicine, University of Technology Sydney, Sydney, New South Wales, Australia. 2St Vincent’s Centre for
Applied Medical Research, St Vincent’s Health Network, Sydney, New South Wales, Australia. 3Decision Neuroscience Laboratory, School of Psychology,
University of New South Wales, Sydney, New South Wales, Australia. 4School of Psychology, University of Sydney, Sydney, New South Wales, Australia.
5These authors contributed equally: Laura A. Bradfield, Beatrice K. Leung. ✉e-mail: laura.bradfield@uts.edu.au; bernard.balleine@unsw.edu.au
The role of the hippocampus in goal-directed action is cur-
rently unclear; studies investigating this issue have produced
contradictory results. Here we reconcile these contradictions
by demonstrating that, in rats, goal-directed action relies
on the dorsal hippocampus, but only transiently, immedi-
ately after initial acquisition. Furthermore, we found that
goal-directed action also depends transiently on physical con-
text, suggesting a psychological basis for the hippocampal
regulation of goal-directed action control.
Evidence regarding the role of the dorsal hippocampus in
goal-directed action is mixed. Neuroimaging studies conducted in
humans have produced several findings suggesting a central role
for the hippocampus in regulating non-navigational goal-directed
decision-making1,2. Likewise, electrophysiological studies in
rodents have implied a role for the dorsal hippocampus in both
navigational3 and non-navigational goal-directed tasks4. Rodent
lesion studies, on the other hand, have found non-navigational
goal-directed actions to be intact despite comprehensive lesions of
the dorsal hippocampus5,6.
One potentially crucial difference between these studies is the
type and amount of training involved. Studies that have detected
a relationship between hippocampal activity and goal-directed
action involved protocols in which participants were either trained
and tested in a single day1 or trained over several days on tasks that
involved multiple contingency switches that could each be encoded
as a novel event2,4. By contrast, studies that found no role for the
hippocampus in goal-directed action trained animals on stable con-
tingencies over multiple days5,6. We sought, therefore, to investigate
the hypothesis that non-navigational goal-directed actions rely on
the dorsal hippocampus but only during initial learning.
We used outcome devaluation tests7 to determine whether
actions were goal directed (Methods). Rats were trained to press two
levers, each delivering a unique outcome: either pellets or sucrose.
After training, the value of one of these outcomes was reduced using
sensory-specific satiety7, after which the rats were given a choice
between the levers in an extinction test (that is, in the absence of
pellet or sucrose delivery). In such tests, rats typically respond more
on the lever associated with the still-valued (non-prefed) outcome
relative to the devalued lever, demonstrating control by both the
current value of the outcome and the action–outcome association
in accordance with definitions of goal-directed action7,8. Our first
series of experiments (Experiments 1–3) investigated whether dor-
sal hippocampal involvement in goal-directed action depends on
the amount of training. To achieve this, we inactivated the dorsal
hippocampus using either local infusions of the GABAA receptor
antagonist muscimol or chemogenetics.
We first assessed the effect of intradorsal hippocampal infu-
sions of muscimol given either before training (Experiment 1a)
or before testing (Experiment 1b; Methods and Extended Data
Fig. 1a–c). For each experiment, rats were first trained to press
both left (A1) and right (A2) levers for polycose (O3) on days 1–5.
On day 6, the left and right levers earned unique outcomes, either
pellets or sucrose (that is, A1→O1 and A2→O2; counterbalanced;
Fig. 1a). Groups did not differ in lever-press acquisition for polycose
or pellets and sucrose in either experiment (largest F value = 1.99,
P = 0.176; Extended Data Fig. 2a–f). On testing, devaluation was
intact (valued > devalued) for rats that received saline infusions
on day 6 but was impaired in muscimol-infused rats (valued =
devalued, Experiment 1a; Fig. 1b), as demonstrated by a group ×
devaluation interaction (F(1,17) = 6.56, P = 0.02) and a significant
simple effect for group SALINE (F(1,17) = 13.46, P = 0.002) but not
group MUSCIMOL (F < 1). On testing, hippocampal inactivation
(Experiment 1b) again disrupted devaluation relative to saline con-
trols (Fig. 1c), as shown by a significant group × devaluation interac-
tion (F(1,14) = 6.09, P = 0.027) and a simple effect in group SALINE
(F(1,14) = 5.86, P = 0.03) but not in group MUSCIMOL (F < 1).
Experiment 2 (design in Fig. 1d) replicated and extended
Experiment 1b, except we omitted polycose pretraining and used
an inhibitory hM4Di DREADD (designer receptor exclusively
activated by designer drug; based on procedures validated previ-
ously9,10) to inactivate the dorsal hippocampus, thus avoiding the
need for multiple infusions. Experiment 2 also sought to establish
the regional specificity of the effect by comparing animals that
received transfection directed toward the CA1 region of the dorsal
hippocampus with a group in which expression was confined to CA2
(Fig. 1e and Extended Data Fig. 1d,e). Half of the rats in each control
group (hM4Di + Vehicle (Veh) and mCherry + clozapine-N-oxide
(CNO)) received viral transfection in CA1 and half received viral
transfection in CA2.
For this experiment, rats were trained to a criterion of a mini-
mum of 20 outcomes on each lever over 1–2 d and then tested for
devaluation performance the following day (Methods). Vehicle and
CNO injections were administered before the test. Initial lever-press
acquisition and the number of outcomes earned on each lever were
similar for all groups (all F values < 1; Extended Data Fig. 2g,h).
On testing, while devaluation was intact in controls (that is, groups
hM4Di + Veh, mCherr y + CNO and hM4Di CA2 + CNO; valued
> devalued), however, it was impaired in the hM4Di CA1 + CNO
Goal-directed actions transiently depend on
dorsal hippocampus
Laura A. Bradfield 1,2,5 ✉ , Beatrice K. Leung3,5, Susan Boldt4, Sophia Liang3 and
Bernard W. Balleine 3 ✉
NATURE NEUROSCIENCE | VOL 23 | OCTOBER 2020 | 1194–1197 | www.nature.com/natureneuroscience
1194
Content courtesy of Springer Nature, terms of use apply. Rights reserved
... Historically, addressing this has been hampered by the very different tasks used to study the HC, which typically focus on spatial information and navigation, versus those applied to the OFC, which normally use nonspatial sensory modalities, especially chemosensory, in simpler Pavlovian or instrumental tasks. Notable exceptions to this dichotomy have shown that the OFC maps spatial relationships in settings normally used to assess HC function [12][13][14][15] and that the HC reflects information and contributes to adaptive behaviors more normally associated with the OFC [16][17][18][19][20][21][22] . When neural activity in the two areas is directly compared in the same task, similarities and differences are evident 10,11,[23][24][25][26][27][28] . ...
Hippocampal output suppresses orbitofrontal cortex schema cell formation
Article
Full-text available
  • Apr 2025
  • NAT NEUROSCI
Both the orbitofrontal cortex (OFC) and the hippocampus (HC) are implicated in the formation of cognitive maps and their generalization into schemas. However, how these areas interact in supporting this function remains unclear, with some proposals supporting a serial model in which the OFC draws on task representations created by the HC to extract key behavioral features and others suggesting a parallel model in which both regions construct representations that highlight different types of information. In the present study, we tested between these two models by asking how schema correlates in rat OFC would be affected by inactivating the output of the HC, after learning and during transfer across problems. We found that the prevalence and content of schema correlates were unaffected by inactivating one major HC output area, the ventral subiculum, after learning, whereas inactivation during transfer accelerated their formation. These results favor the proposal that the OFC and HC operate in parallel to extract different features defining cognitive maps and schemas.
View
... The impairment in goal-directed response in iCKO mice may be attributable to altered hippocampal activity. The dorsal hippocampus is transiently involved in the formation of action-outcome associations [57,58], which could have prevented mice from forming the associations necessary to guide their behavior in the devaluation test. Another possibility is that the altered interneuron migration induced in iCKO mice misplaced striatal interneurons [20]. ...
Dysregulation of neuropilin-2 expression in inhibitory neurons impairs hippocampal circuit development and enhances risk for autism-related behaviors and seizures
Article
Full-text available
  • Nov 2024
Dysregulation of development, migration, and function of interneurons, collectively termed interneuronopathies, have been proposed as a shared mechanism for autism spectrum disorders (ASDs) and childhood epilepsy. Neuropilin-2 (Nrp2), a candidate ASD gene, is a critical regulator of interneuron migration from the median ganglionic eminence (MGE) to the pallium, including the hippocampus. While clinical studies have identified Nrp2 polymorphisms in patients with ASD, whether selective dysregulation of Nrp2-dependent interneuron migration contributes to pathogenesis of ASD and enhances the risk for seizures has not been evaluated. We tested the hypothesis that the lack of Nrp2 in MGE-derived interneuron precursors disrupts the excitation/inhibition balance in hippocampal circuits, thus predisposing the network to seizures and behavioral patterns associated with ASD. Embryonic deletion of Nrp2 during the developmental period for migration of MGE derived interneuron precursors (iCKO) significantly reduced parvalbumin, neuropeptide Y, and somatostatin positive neurons in the hippocampal CA1. Consequently, when compared to controls, the frequency of inhibitory synaptic currents in CA1 pyramidal cells was reduced while frequency of excitatory synaptic currents was increased in iCKO mice. Although passive and active membrane properties of CA1 pyramidal cells were unchanged, iCKO mice showed enhanced susceptibility to chemically evoked seizures. Moreover, iCKO mice exhibited selective behavioral deficits in both preference for social novelty and goal-directed learning, which are consistent with ASD-like phenotype. Together, our findings show that disruption of developmental Nrp2 regulation of interneuron circuit establishment, produces ASD-like behaviors and enhanced risk for epilepsy. These results support the developmental interneuronopathy hypothesis of ASD epilepsy comorbidity.
View
... Specifically, half of the mice from each group were trained to press a left lever for pellets and a right lever for a 20 % sucrose solution, and the other half received the opposite arrangement. Because we have previously demonstrated that outcome devaluation performance is dependent on the dorsal hippocampus only during the early stages of learning, becoming hippocampally-independent after approximately 6 days of lever press training (Bradfield et al., 2020;Dhungana et al., 2023), we first tested outcome devaluation after 4 days of training ("Day-4 Test") so as not to miss this critical window. For this test, mice were given 1 hr of unrestricted access to one of the two outcomes so that they could consume it to satiety to reduce its value (devalued outcome) relative to the other outcome (valued outcome) (Balleine and Dickinson, 1998). ...
View
... HC is another critical brain region implicated in encoding the task state, however the distinct contributions of the HC and the OFC to the cognitive map remain unclear. Early research indicates that HC neurons encode animals' current location 1,10 and guide goal-directed navigation [11][12][13] . HC is also crucial for episodic memory 14,15 , and HC neuron activity differs across various contexts 16,17 , reflecting the situation or environment in which a series of events occur. ...
Expectancy-related changes in firing of dopamine neurons depend on hippocampus
Article
Full-text available
  • Oct 2024
The orbitofrontal cortex (OFC) and hippocampus (HC) both contribute to the cognitive maps that support flexible behavior. Previously, we used the dopamine neurons to measure the functional role of OFC. We recorded midbrain dopamine neurons as rats performed an odor-based choice task, in which expected rewards were manipulated across blocks. We found that ipsilateral OFC lesions degraded dopaminergic prediction errors, consistent with reduced resolution of the task states. Here we have repeated this experiment in male rats with ipsilateral HC lesions. The results show HC also shapes the task states, however unlike OFC, which provides information local to the trial, the HC is necessary for estimating upper-level hidden states that distinguish blocks. The results contrast the roles of the OFC and HC in cognitive mapping and suggest that the dopamine neurons access rich information from distributed regions regarding the environment’s structure, potentially enabling this teaching signal to support complex behaviors.
View
... It receives a large amount of visuospatial information projections, including from the dorsolateral entorhinal cortex and caudal entorhinal cortex, which in turn receive primary inputs from the perirhinal cortex and postrhinal cortex. 17 In brief, the dorsal hippocampus-hypothalamus complex, together with the retrosplenial cortex and anterior cingulate cortex, forms an important cortical network primarily responsible for cognitive processes such as spatial learning, memory, navigation, and exploration. The ventral hippocampus has strong fiber connections with subcortical structures such as the rostral hypothalamus and amygdala, which are closely related to its functions. ...
Research progress on hippocampal neurogenesis in autism spectrum disorder
Article
Full-text available
  • Jul 2024
Autism spectrum disorder (ASD) is a group of severe neurodevelopmental disorders with unclear etiology and significant heterogeneity that is emerging as a global public health concern. Increasing research suggests the involvement of hippocampal neurogenesis defects in the onset and development of ASD, drawing increasing amounts of attention to hippocampal neurogenesis issues in ASD. In this paper, we analyze relevant international studies on hippocampal neurogenesis in ASD, discuss the role of neurobiology in the pathogenesis of ASD, and explore the potential of improving hippocampal neurogenesis as a therapeutic approach for ASD. This review aims to provide new treatment perspectives and theoretical foundations for clinical practice.
View
... The impairment in goal-directed response in iCKO mice may be attributable to altered hippocampal activity. The dorsal hippocampus is transiently involved in the formation of action-outcome associations (45,46), which could have prevented mice from forming the associations necessary to guide their behavior in the devaluation test. Another possibility is that the altered interneuron migration induced in iCKO mice misplaced striatal interneurons (20). ...
Dysregulation of Neuropilin-2 Expression in Inhibitory Neurons Impairs Hippocampal Circuit Development Leading to Autism-Epilepsy Phenotype
Preprint
Full-text available
  • Feb 2024
Dysregulation of development, migration, and function of interneurons, collectively termed interneuronopathies, have been proposed as a shared mechanism for autism spectrum disorders (ASDs) and childhood epilepsy. Neuropilin-2 (Nrp2), a candidate ASD gene, is a critical regulator of interneuron migration from the median ganglionic eminence (MGE) to the pallium, including the hippocampus. While clinical studies have identified Nrp2 polymorphisms in patients with ASD, whether dysregulation of Nrp2-dependent interneuron migration contributes to pathogenesis of ASD and epilepsy has not been tested. We tested the hypothesis that the lack of Nrp2 in MGE-derived interneuron precursors disrupts the excitation/inhibition balance in hippocampal circuits, thus predisposing the network to seizures and behavioral patterns associated with ASD. Embryonic deletion of Nrp2 during the developmental period for migration of MGE derived interneuron precursors (iCKO) significantly reduced parvalbumin, neuropeptide Y, and somatostatin positive neurons in the hippocampal CA1. Consequently, when compared to controls, the frequency of inhibitory synaptic currents in CA1 pyramidal cells was reduced while frequency of excitatory synaptic currents was increased in iCKO mice. Although passive and active membrane properties of CA1 pyramidal cells were unchanged, iCKO mice showed enhanced susceptibility to chemically evoked seizures. Moreover, iCKO mice exhibited selective behavioral deficits in both preference for social novelty and goal-directed learning, which are consistent with ASD-like phenotype. Together, our findings show that disruption of developmental Nrp2 regulation of interneuron circuit establishment, produces ASD-like behaviors and enhanced risk for epilepsy. These results support the developmental interneuronopathy hypothesis of ASD epilepsy comorbidity.
View
Region- and stage-specific cortical plasticity shapes the multifaceted process of habit formation
Preprint
  • May 2025
Habits are crucial for shaping our normal and pathological behaviors. They emerge through a transition in decision-making strategy, from flexible adaptation to current demands (goal-directed behavior) to automatic repeats of previous behaviors (habitual behavior). This transition of decision-making strategy induces changes in an underlying cognitive process for selecting an action while superficially maintaining the execution of the selected action. However, the neurocircuit mechanisms supporting these changes remain unexplored. By developing a behavioral paradigm capable of inducing this transition within a defined time window, we performed a longitudinal analysis of the neuronal circuits involved in this process using ex vivo electrophysiology, optogenetic erasure of synaptic plasticity and in vivo Ca2+-imaging. We found that the decision-making strategy and action execution are orthogonally controlled by plasticity in distinct cortical pathways. Erasure of these changes independently alter the decision-making strategy or the action execution without influencing each other, supporting a dichotomous circuit mechanism. This study underscores the critical role of pathway-specific cortical plasticity in regulating the multifaceted process shaping habitual behavior, offering valuable insights into the pathological mechanisms of maladaptive habits, such as compulsive disorders.
View
View
View
View
Complementary Task Structure Representations in Hippocampus and Orbitofrontal Cortex during an Odor Sequence Task
Article
Full-text available
  • Oct 2019
  • CURR BIOL
Both hippocampus (HPC) and orbitofrontal cortex (OFC) have been shown to be critical for behavioral tasks that require use of an internal model or cognitive map, composed of the states and the relationships between them, which define the current environment or task at hand. One general idea is that the HPC provides the cognitive map, which is then transformed by OFC to emphasize information of relevance to current goals. Our previous analysis of ensemble activity in OFC in rats performing an odor sequence task revealed a rich representation of behaviorally relevant task structure, consistent with this proposal. Here, we compared those data to recordings from single units in area CA1 of the HPC of rats performing the same task. Contrary to expectations that HPC ensembles would represent detailed, even incidental, information defining the full task space, we found that HPC ensembles-like those in OFC-failed to distinguish states when it was not behaviorally necessary. However, hippocampal ensembles were better than those in OFC at distinguishing task states in which prospective memory was necessary for future performance. These results suggest that, in familiar environments, the HPC and OFC may play complementary roles, with the OFC maintaining the subjects' current position on the cognitive map or state space, supported by HPC when memory demands are high.
View
Sequential replay of nonspatial task states in the human hippocampus
Article
Full-text available
  • Jun 2019
Replay of activity in the human brain Electrophysiological recordings in rats and mice have shown that specific hippocampal neuronal activity patterns are sequentially reactivated during rest periods or sleep. Does the human hippocampus also replay activity sequences, even in a nonspatial task, such as, for example, decision-making? Schuck and Niv studied functional magnetic resonance imaging signals in subjects after they had learned a decision-making task. While people rested, the replay of activity patterns in the hippocampus reflected the order of previous task-state sequences. Thus, sequential hippocampal reactivation might participate in decision-making in humans. Science , this issue p. eaaw5181
View
Extinction Generates Outcome-Specific Conditioned Inhibition
Article
Full-text available
  • Oct 2016
Extinction involves altering a previously established predictive relationship between a cue and its outcome by repeatedly presenting that cue alone. Although it is widely accepted that extinction generates some form of inhibitory learning [1–4], direct evidence for this claim has been lacking, and the nature of the associative changes induced by extinction have, therefore, remained a matter of debate [5–8]. In the current experiments, we used a novel behavioral approach that we recently developed and that provides a direct measure of conditioned inhibition [9] to compare the influence of extinguished and non-extinguished cues on choice between goal-directed actions. Using this approach, we provide direct evidence that extinction generates outcome-specific conditioned inhibition. Furthermore, we demonstrate that this inhibitory learning is controlled by the infralimbic cortex (IL); inactivation of the IL using M4 DREADDs abolished outcome-specific inhibition and rendered the cue excitatory. Importantly, we found that context modulated this inhibition. Outside its extinction context, the cue was excitatory and functioned as a specific predictor of its previously associated outcome, biasing choice toward actions earning the same outcome. In its extinction context, however, the cue acted as a specific inhibitor and biased choice toward actions earning different outcomes. Context modulation of these excitatory and inhibitory memories was mediated by the dorsal hippocampus (HPC), suggesting that the HPC and IL act in concert to control the influence of conditioned inhibitors on choice. These findings demonstrate for the first time that extinction turns a cue into a net inhibitor that can influence choice via counterfactual action-outcome associations.
View
Contextual Control of Instrumental Actions and Habits
Article
Full-text available
  • Nov 2014
After a relatively small amount of training, instrumental behavior is thought to be an action under the control of the motivational status of its goal or reinforcer. After more extended training, behavior can become habitual and insensitive to changes in reinforcer value. Recently, instrumental responding has been shown to weaken when tested outside of the training context. The present experiments compared the sensitivity of instrumental responding in rats with a context switch after training procedures that might differentially generate actions or habits. In Experiment 1, lever pressing was decremented in a new context after either short, medium, or long periods of training on either random-ratio or yoked random-interval reinforcement schedules. Experiment 2 found that more minimally trained responding was also sensitive to a context switch. Moreover, Experiment 3 showed that when the goal-directed component of responding was removed by devaluing the reinforcer, the residual responding that remained was still sensitive to the change of context. Goal-directed responding, in contrast, transferred across contexts. Experiment 4 then found that after extensive training, a habit that was insensitive to reinforcer devaluation was still decremented by a context switch. Overall, the results suggest that a context switch primarily influences instrumental habit rather than action. In addition, even a response that has received relatively minimal training may have a habit component that is insensitive to reinforcer devaluation but sensitive to the effects of a context switch.
View
The why, what, where, when and how of goal-directed choice: Neuronal and computational principles
Article
Full-text available
The central problems that goal-directed animals must solve are: 'What do I need and Why, Where and When can this be obtained, and How do I get it?' or the H4W problem. Here, we elucidate the principles underlying the neuronal solutions to H4W using a combination of neurobiological and neurorobotic approaches. First, we analyse H4W from a system-level perspective by mapping its objectives onto the Distributed Adaptive Control embodied cognitive architecture which sees the generation of adaptive action in the real world as the primary task of the brain rather than optimally solving abstract problems. We next map this functional decomposition to the architecture of the rodent brain to test its consistency. Following this approach, we propose that the mammalian brain solves the H4W problem on the basis of multiple kinds of outcome predictions, integrating central representations of needs and drives (e.g. hypothalamus), valence (e.g. amygdala), world, self and task state spaces (e.g. neocortex, hippocampus and prefrontal cortex, respectively) combined with multi-modal selection (e.g. basal ganglia). In our analysis, goal-directed behaviour results from a well-structured architecture in which goals are bootstrapped on the basis of predefined needs, valence and multiple learning, memory and planning mechanisms rather than being generated by a singular computation.
View
Hippocampal Contributions to Model-Based Planning and Spatial Memory
Article
  • Mar 2019
Little is known about the neural mechanisms that allow humans and animals to plan actions using knowledge of task contingencies. Emerging theories hypothesize that it involves the same hippocampal mechanisms that support self-localization and memory for locations. Yet limited direct evidence supports the link between planning and the hippocampal place map. We addressed this by investigating model-based planning and place memory in healthy controls and epilepsy patients treated using unilateral anterior temporal lobectomy with hippocampal resection. Both functions were impaired in the patient group. Specifically, the planning impairment was related to right hippocampal lesion size, controlling for overall lesion size. Furthermore, although planning and boundary-driven place memory covaried in the control group, this relationship was attenuated in patients, consistent with both functions relying on the same structure in the healthy brain. These findings clarify both the neural mechanism of model-based planning and the scope of hippocampal contributions to behavior. Testing patients with hippocampal damage, Vikbladh et al. demonstrate that model-based planning and place memory rely on a common hippocampal substrate. The study bridges the reinforcement learning and spatial memory literatures to clarify the scope of hippocampal contributions to behavior.
View
Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework
Article
  • Jan 2017
Wereview the psychology and neuroscience of reinforcement learning (RL), which has experienced significant progress in the past two decades, enabled by the comprehensive experimental study of simple learning and decisionmaking tasks. However, one challenge in the study of RL is computational: The simplicity of these tasks ignores important aspects of reinforcement learning in the real world: (a) State spaces are high-dimensional, continuous, and partially observable; this implies that (b) data are relatively sparse and, indeed, precisely the same situation may never be encountered twice; furthermore, (c) rewards depend on the long-term consequences of actions in ways that violate the classical assumptions that make RL tractable. A seemingly distinct challenge is that, cognitively, theories of RL have largely involved procedural and semantic memory, the way in which knowledge about action values or world models extracted gradually from many experiences can drive choice. This focus on semantic memory leaves out many aspects of memory, such as episodic memory, related to the traces of individual events. We suggest that these two challenges are related. The computational challenge can be dealt with, in part, by endowing RL systems with episodic memory, allowing them to (a) efficiently approximate value functions over complex state spaces, (b) learn with very little data, and (c) bridge long-term dependencies between actions and rewards. We review the computational theory underlying this proposal and the empirical evidence to support it. Our proposal suggests that the ubiquitous and diverse roles of memory in RL may function as part of an integrated learning system. Expected final online publication date for the Annual Review of Psychology Volume 68 is January 03, 2017. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
View
View
Medial Orbitofrontal Cortex Mediates Outcome Retrieval in Partially Observable Task Situations
Article
  • Nov 2015
Choice between actions often requires the ability to retrieve action consequences in circumstances where they are only partially observable. This capacity has recently been argued to depend on orbitofrontal cortex; however, no direct evidence for this hypothesis has been reported. Here, we examined whether activity in the medial orbitofrontal cortex (mOFC) underlies this critical determinant of decision-making in rats. First, we simulated predictions from this hypothesis for various tests of goal-directed action by removing the assumption that rats could retrieve partially observable outcomes and then tested those predictions experimentally using manipulations of the mOFC. The results closely followed predictions; consistent deficits only emerged when action consequences had to be retrieved. Finally, we put action selection based on observable and unobservable outcomes into conflict and found that whereas intact rats selected actions based on the value of retrieved outcomes, mOFC rats relied solely on the value of observable outcomes.
View
Hippocampal theta sequences reflect current goals
Article
  • Jan 2015
Hippocampal information processing is discretized by oscillations, and the ensemble activity of place cells is organized into temporal sequences bounded by theta cycles. Theta sequences represent time-compressed trajectories through space. Their forward-directed nature makes them an intuitive candidate mechanism for planning future trajectories, but their connection to goal-directed behavior remains unclear. As rats performed a value-guided decision-making task, the extent to which theta sequences projected ahead of the animal's current location varied on a moment-by-moment basis depending on the rat's goals. Look-ahead extended farther on journeys to distant goals than on journeys to more proximal goals and was predictive of the animal's destination. On arrival at goals, however, look-ahead was similar regardless of where the animal began its journey from. Together, these results provide evidence that hippocampal theta sequences contain information related to goals or intentions, pointing toward a potential spatial basis for planning.
View