ArticlePublisher preview available

Past experience shapes the neural circuits recruited for future learning

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

Experimental research controls for past experience, yet prior experience influences how we learn. Here, we tested whether we could recruit a neural population that usually encodes rewards to encode aversive events. Specifically, we found that GABAergic neurons in the lateral hypothalamus (LH) were not involved in learning about fear in naïve rats. However, if these rats had prior experience with rewards, LH GABAergic neurons became important for learning about fear. Interestingly, inhibition of these neurons paradoxically enhanced learning about neutral sensory information, regardless of prior experience, suggesting that LH GABAergic neurons normally oppose learning about irrelevant information. These experiments suggest that prior experience shapes the neural circuits recruited for future learning in a highly specific manner, reopening the neural boundaries we have drawn for learning of particular types of information from work in naïve subjects.
Rats in our NpHR learners group showed a persistent increase in conditioned fear to the contextual cues, which extinguished before tone presentations in the extinction test During conditioning, our NpHR learners group showed high levels of fear to the background contextual cues (left; see Fig. 3 in main text for more information). To reduce these levels of context fear before the test, 24 hours after conditioning, rats received a context extinction session where they were placed in the experimental chambers without any stimuli. Here, we found that our NpHR learners group maintained higher level of context fear relative to our eYFP learners group (middle). This context extinction was effective in reducing contextual fear as all rats showed low levels of freezing at the beginning of the next test session, where we presented the tone under extinction to examine fear that had acquired to these stimuli (right). A mixed-design repeated-measures ANOVA on levels of freezing to the contextual cues across the context and tone extinction sessions showed a main effect of time (F14,140 = 4.614, p = 0.000), and a significant session x time x group interaction (F14,140 = 1.697, p = 0.032). This interaction was owed to a between-group difference in freezing during context extinction that revealed itself most prominently towards the end of the scoring period (group: F1,10 = 5.939, p = 0.035), that was not seen in the tone extinction test (group: F1,10 = 0.007, p = 1.000). Further, there was a significant difference in freezing exhibited by the NpHR group when comparing the context extinction session with the tone test (n = 6 rats; F1,10 = 8.071, p = 0.018), that was not present in the eYFP control group (n = 6 rats; F1,10 = 1.161, p = 0.307). Finally, a one-way ANOVA showed there was no between-group difference in freezing to the context immediately before tone presentations in the tone test after context extinction had taken place (F1,10 = 1.943, p = 0.194). Error bars = SEM.
… 
This content is subject to copyright. Terms and conditions apply.
Articles
https://doi.org/10.1038/s41593-020-00791-4
1Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA. 2National Institute on Drug Abuse, Intramural Program, Baltimore,
MD, USA. e-mail: melissa.j.sharpe@psych.ucla.edu; geoffrey.schoenbaum@nih.gov
Prior experience shapes the way we view our world. War mov-
ies are rife with veterans jumping at the sound of a car door
slamming when they return home, and lottery winners may
view life through rose-tinted glasses. We use our prior experience to
build models of our environment, and preferentially interpret new
experiences in a manner consistent with these models13. Despite
this, experimental research often does not consider a role for prior
experience as a factor. Research involving the human population
samples from homogeneous groups, controlling factors that might
give rise to differences. And of course, one of the main benefits of
animal research is an ability to eliminate prior experience. Rats and
mice, and to a lesser extent primates, used for experimental research
are housed in experiential vacuums, devoid of any exposure to
events that may contaminate their naïvety.
The few paradigms that investigated prior experience as an
experimental variable focused on extreme cases, where humans
or other animals are exposed to traumatic events47. Here, prior
trauma appears to ‘prime’ fear circuits to learn, resulting in indi-
viduals who have experienced trauma learning about an aversive
event so mild that others would usually ignore it4,5,8. This work is
integral to understanding some of the factors that could contrib-
ute to pathological fear, such as that seen in post-traumatic stress
disorder (PTSD). But what if these impacts can occur outside of a
pathological environment?
Here, we focused on GABAergic neurons in the LH, which
are generally implicated in learning about rewards913, to test
whether nonaversive experiences might prime other circuits to
encode fear memories. We found that LH GABAergic neurons
were important for encoding fear memories, but only in rats that
had experience with reward learning. Interestingly, reward learn-
ing did not recruit LH GABAergic neurons to encode all types of
learning; inhibiting these neurons while rats learned about cue–
cue relationships paradoxically enhanced such learning, regard-
less of whether the rats had experience with reward learning. This
suggests that these neurons oppose the development of associa-
tions between cues regardless of prior experience, biasing learning
towards rewards, but that they can be recruited to learn about other
motivationally relevant events, like aversive events, with appropriate
prior experience.
Results
Experiment 1: GABAergic neurons in the LH are not necessary to
learn about aversive events in naïve rats. Before training, all rats
underwent surgery to infuse virus and implant fiber optics targeting
the LH (Fig. 1 and Extended Data Fig. 1). We infused a Cre-dependent
adeno-associated virus (AAV) carrying halorhodopsin (NpHR)
(AAV5-EF1α-DIO-eNpHR3.0-eYFP; NpHR experimental group;
n = 4) or the control AAV vector (AAV5-EF1α-DIO-eYFP; eYFP
control group; n = 4) fused to enhanced yellow fluorescent protein
into the LH of rats expressing Cre recombinase from the glutamate
decarboxylase 1 (GAD1) promoter—one of two genes encoding the
GAD enzyme that converts glutamate to GABA (hereafter referred
to a GAD-Cre rats)10. During this surgery, we also implanted optic
fibers terminating 0.5 mm above the injection site in the LH, allow-
ing us to silence GABAergic neurons in the LH by delivering green
light (532 nm and 16 mW).
During conditioning, rats received three tone-shock pairings,
with an intertrial interval (ITI) varying around a 7-min mean. A
10-s auditory tone (70 dB) was presented followed by a 1-s foot
shock (0.5 mA), after a 1-s delay. Light was delivered into the brain
during the tone, beginning 500 ms before tone onset, and continu-
ing until 500 ms after tone offset. During this conditioning, all rats
learned to fear the tone stimulus, exhibiting an increase in freezing
elicited by the tone, with no difference between groups (Fig. 2a; trial:
F2,12 = 5.812, P = 0.017; trial × group: F2,12 = 0.187, P = 0.831; group:
F1,6 = 0.377, P = 0.562). There were no differences in levels of freez-
ing exhibited to the background contextual cues before cue pre-
sentation (mean (s.e.m.): eYFP, 45% (22.17%); NpHR, 35% (15%);
F1,6 = 0.140, P = 0.722). Following conditioning, we tested whether
inactivation of LH GABAergic neurons during conditioning would
impact freezing the following day. To test this, rats were presented
with the tone again without shock or light delivery into the LH. Rats
exhibited high levels of fear in response to the tone, and the level of
freezing did not differ between groups (Fig. 2b; group: F1,6 = 0.206,
Past experience shapes the neural circuits
recruited for future learning
Melissa J. Sharpe 1 ✉ , Hannah M. Batchelor2, Lauren E. Mueller2, Matthew P. H. Gardner2 and
Geoffrey Schoenbaum 2 ✉
Experimental research controls for past experience, yet prior experience influences how we learn. Here, we tested whether we
could recruit a neural population that usually encodes rewards to encode aversive events. Specifically, we found that GABAergic
neurons in the lateral hypothalamus (LH) were not involved in learning about fear in naïve rats. However, if these rats had prior
experience with rewards, LH GABAergic neurons became important for learning about fear. Interestingly, inhibition of these
neurons paradoxically enhanced learning about neutral sensory information, regardless of prior experience, suggesting that LH
GABAergic neurons normally oppose learning about irrelevant information. These experiments suggest that prior experience
shapes the neural circuits recruited for future learning in a highly specific manner, reopening the neural boundaries we have
drawn for learning of particular types of information from work in naïve subjects.
NATURE NEUROSCIENCE | VOL 24 | MARCH 2021 | 391–400 | www.nature.com/natureneuroscience 391
Content courtesy of Springer Nature, terms of use apply. Rights reserved
... Promisingly, the modern neuroscientists' toolbox permits unprecedented access to neural populations and circuits with incredible neurochemical and spatiotemporal specificity (see Navabpour et al., 2020;Nectow & Nestler, 2020;Sternson & Roth, 2014 for reviews). These current technologies, such as chemogenetics, optogenetics, and in vivo fiber photometry, are commonly applied to Pavlovian fear conditioning paradigms (see J. Ma et al., 2021;Sharpe et al., 2021;Venkataraman et al., 2021 for examples) to expand our understanding of emotion, learning, and memory neurobiology. For example, the amygdala's role in Pavlovian conditioning has been appreciated for decades (Blanchard & Blanchard, 1972;Kellicutt & Schwartzbaum, 1963), yet modern approaches have only recently highlighted particular amygdaloid cell types involved (Barsy et al., 2020;Gafford & Ressler, 2016). ...
Thesis
Full-text available
Fear-related psychopathologies, such as post-traumatic stress disorder, are strongly linked to aberrant neural circuits governing learning, memory, and emotion. Unexplored cells in the lateral hypothalamus (LH) and zona incerta (ZI) that express the relaxin-3 peptide receptor, RXFP3, are putatively involved in these processes. Anterograde tracing from this population revealed projections to the periaqueductal gray, lateral habenula, and lateral septum, which are implicated in associative learning, context discrimination, and control of defensive behaviours. Thus, it is plausible that these cells are involved in context-dependent Pavlovian fear conditioning and extinction. To interrogate this possibility, RXFP3-Cre mice were bilaterally injected with a Cre-dependent excitatory DREADD (hM3Dq) or mCherry control virus into the LH/ZI. Mice were subjected to a three-day context-dependent fear conditioning protocol involving tone-footshock (CS-US) conditioning (Day 1), fear extinction (Day 2), and recall test (Day 3). RXFP3+ LH/ZI neurons were activated by injection of CNO (3 mg/kg, i.p.) during fear extinction, which decreased freezing to the CS in extinction context B, but not extinction context A. Interestingly, activating these cells also enhanced general locomotion and caused escape-like jumping. Subsequent analysis revealed that jumping did not significantly drive attenuated freezing. However, it remains unclear if this result reflected changes in context-dependent fear expression or increased arousal. Additionally, activating RXFP3+ LH/ZI cells during extinction did not affect extinction memory retrieval or fear renewal. Though this suggests that these cells may not have been involved in extinction memory encoding, this result may have indicated issues with strainspecific contextual discrimination. Additionally, brain-wide Fos expression analysis revealed that activating RXFP3+ LH/ZI neurons increased coordinated activity in numerous stress, arousal, learning, and memory implicated regions. Overall, this study provided novel information regarding RXFP3+ LH/ZI cells’ involvement in Pavlovian fear conditioning and extinction, and highlighted the putative role of these cells in various aspects of stress, learning, and memory.
... During pre-exposure, we delivered light into the brain (470 nm, 1 s, 20 Hz) at the onset of the cue. We have previously used a greater number of pre-exposure trials to generate successful latent inhibition 58 . We used less pre-exposure here as we wanted to give an opportunity to see either an enhancement or reduction in latent inhibition in our experimental group. ...
Article
Full-text available
Studies investigating the neural mechanisms by which associations between cues and predicted outcomes control behavior often use associative learning frameworks to understand the neural control of behavior. These frameworks do not always account for the full range of effects that novelty can have on behavior and future associative learning. Here, in mice, we show that dopamine in the nucleus accumbens core is evoked by novel, neutral stimuli, and the trajectory of this response over time tracked habituation to these stimuli. Habituation to novel cues before associative learning reduced future associative learning, a phenomenon known as latent inhibition. Crucially, trial-by-trial dopamine response patterns tracked this phenomenon. Optogenetic manipulation of dopamine responses to the cue during the habituation period bidirectionally influenced future associative learning. Thus, dopamine signaling in the nucleus accumbens core has a causal role in novelty-based learning in a way that cannot be predicted based on purely associative factors.
... 56,57,[60][61][62][63] To expand, not all co-occurring events need be associated, and there are also regions (e.g., lateral hypothalamus) whose function appears to be opposing the learning of relationships that do not immediately predict rewards. 64,65 Such findings situate the VTA DA prediction error at the center of a dynamic system whose main function is to direct learning in one way or another via distinct circuits, depending on current context or motivational state, and past experience. Future research will tell how far we can push the boundaries of dopamine's involvement in learning and cognition. ...
Article
For over two decades, phasic activity in midbrain dopamine neurons was considered synonymous with the prediction error in temporal-difference reinforcement learning.[1], [2], [3], [4] Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received, allowing for the calculation of prediction errors.⁵,⁶ Phasic firing of dopamine neurons was proposed to reflect this computation,¹,² facilitating the backpropagation of value from the predicted reward to the reward-predictive stimulus, thus reducing future prediction errors. There are two critical assumptions of this proposal: (1) that dopamine errors can only facilitate learning about scalar value and not more complex features of predicted rewards, and (2) that the dopamine signal can only be involved in anticipatory cue-reward learning in which cues or actions precede rewards. Recent work[7], [8], [9], [10], [11], [12], [13], [14], [15] has challenged the first assumption, demonstrating that phasic dopamine signals across species are involved in learning about more complex features of the predicted outcomes, in a manner that transcends this value computation. Here, we tested the validity of the second assumption. Specifically, we examined whether phasic midbrain dopamine activity would be necessary for backward conditioning—when a neutral cue reliably follows a rewarding outcome.[16], [17], [18], [19], [20] Using a specific Pavlovian-to-instrumental transfer (PIT) procedure,[21], [22], [23] we show rats learn both excitatory and inhibitory components of a backward association, and that this association entails knowledge of the specific identity of the reward and cue. We demonstrate that brief optogenetic inhibition of VTADA neurons timed to the transition between the reward and cue reduces both of these components of backward conditioning. These findings suggest VTADA neurons are capable of facilitating associations between contiguously occurring events, regardless of the content of those events. We conclude that these data may be in line with suggestions that the VTADA error acts as a universal teaching signal. This may provide insight into why dopamine function has been implicated in myriad psychological disorders that are characterized by very distinct reinforcement-learning deficits.
... We propose that understanding cognitive processes will require considering the wider set of functions an animal is trying to optimize, beyond the computation of interest in a particular study. To understand how long-ago cognitive experience and current cognitive demands set up these different neural circuit landscapes for outwardly identical navigation decisions, we suggest to carefully control for and to intentionally vary cognitive experience in laboratory settings (Plitt and Giocomo, 2021;Sharpe et al., 2021), thereby approximating more naturalistic scenarios. We anticipate this approach will be particularly powerful to illuminate neural circuit differences underlying interindividual variability and changes in neural dynamics during learning (Oby et al., 2019;Sadtler et al., 2014). ...
Article
Full-text available
Neural activity in the mammalian cortex has been studied extensively during decision tasks, and recent work aims to identify under what conditions cortex is actually necessary for these tasks. We discovered that mice with distinct cognitive experiences, beyond sensory and motor learning, use different cortical areas and neural activity patterns to solve the same navigation decision task, revealing past learning as a critical determinant of whether cortex is necessary for goal-directed navigation. We used optogenetics and calcium imaging to study the necessity and neural activity of multiple cortical areas in mice with different training histories. Posterior parietal cortex and retrosplenial cortex were mostly dispensable for accurate performance of a simple navigation task. In contrast, these areas were essential for the same simple task when mice were previously trained on complex tasks with delay periods or association switches. Multi-area calcium imaging showed that, in mice with complex-task experience, single-neuron activity had higher selectivity and neuron-neuron correlations were weaker, leading to codes with higher task information. Therefore, past experience is a key factor in determining whether cortical areas have a causal role in goal-directed navigation.
... In other words, nodes represent anatomically and functionally distinct ensembles of cells that integrate and process neural information from many different inputs to dictate the output of that circuit. These complex networks of connectivity are responsible for everything from processing sensory information (Lohse et al., 2020), attaching valence to stimuli that drive motivated behaviors (Reynolds and Berridge, 2002;Root et al., 2014;Al-Hasani et al., 2015), maintaining physiological homeostasis (Atasoy et al., 2012;Madden and Morrison, 2019), learning, and consolidating memories (Ruder et al., 2021;Sharpe et al., 2021). ...
Article
Full-text available
Neural circuits and the cells that comprise them represent the functional units of the brain. Circuits relay and process sensory information, maintain homeostasis, drive behaviors, and facilitate cognitive functions such as learning and memory. Creating a functionally-precise map of the mammalian brain requires anatomically tracing neural circuits, monitoring their activity patterns, and manipulating their activity to infer function. Advancements in cell-type-specific genetic tools allow interrogation of neural circuits with increased precision. This review provides a broad overview of recombination-based and activity-driven genetic targeting approaches, contemporary viral tracing strategies, electrophysiological recording methods, newly developed calcium, and voltage indicators, and neurotransmitter/neuropeptide biosensors currently being used to investigate circuit architecture and function. Finally, it discusses methods for acute or chronic manipulation of neural activity, including genetically-targeted cellular ablation, optogenetics, chemogenetics, and over-expression of ion channels. With this ever-evolving genetic toolbox, scientists are continuing to probe neural circuits with increasing resolution, elucidating the structure and function of the incredibly complex mammalian brain.
... Indeed, an excess of subcortical dopamine (a trademark of schizophrenia) would be expected to be correlated with an excess in learning relationships between potentially irrelevant events-which could result in hallucinogenic or delusional experiences [52][53][54][55][56][57] . To expand, not all co-occurring events need be associated, and there are also regions (e.g., lateral hypothalamus) whose function appears to be opposing the learning of relationships that do not immediately predict rewards 58,59 . Such findings situate the VTADA prediction error at the center of a dynamic system whose main function is to direct learning in one way or another via distinct circuits, depending on current context or motivational state, and past experience. ...
Preprint
Full-text available
For over two decades, midbrain dopamine was considered synonymous with the prediction error in temporal-difference reinforcement learning. Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received allowing for the calculation of prediction errors. Phasic firing of dopamine neurons was proposed to reflect this computation, facilitating the backpropagation of value from the predicted reward to the reward-predictive stimulus, thus reducing future prediction errors. There are two critical assumptions of this proposal: 1) that dopamine errors can only facilitate learning about scalar value and not more complex features of predicted rewards, and 2) that the dopamine signal can only be involved in anticipatory learning in which cues or actions precede rewards. Recent work has challenged the first assumption, demonstrating that phasic dopamine signals across species are involved in learning about more complex features of the predicted outcomes, in a manner that transcends this value computation. Here, we tested the validity of the second assumption. Specifically, we examined whether phasic midbrain dopamine activity would be necessary for backward conditioning—when a neutral cue reliably follows a rewarding outcome. Using a specific Pavlovian-to-Instrumental Transfer (PIT) procedure, we show rats learn both excitatory and inhibitory components of a backward association, and that this association entails knowledge of the specific identity of the reward and cue. We demonstrate that brief optogenetic inhibition of VTA DA neurons timed to the transition between the reward and cue, reduces both of these components of backward conditioning. These findings suggest VTA DA neurons are capable of facilitating associations between contiguously occurring events, regardless of the content of those events. We conclude that these data are in line with suggestions that the VTA DA error acts as a universal teaching signal. This may provide insight into why dopamine function has been implicated in a myriad of psychological disorders that are characterized by very distinct reinforcement-learning deficits. Graphical Abstract
... Related to our motivating examples of crisis experiences, recent experimental ndings show that a neural population that usually encodes rewards can start encoding adverse events (Sharpe et al. 2021). In that way, past experiences shape the neural circuits recruited for future learning in a highly speci c manner, implying that prior experiences might affect the process of learning in and of itself. ...
Article
Personal experiences of economic outcomes, from global financial crises to individual-level job losses, can shape individual beliefs, risk attitudes, and choices for years to come. A growing literature on experience effects shows that individuals act as if past outcomes that they experienced were overly likely to occur again, even if they are fully informed about the actual likelihood. This reaction to past experiences is long-lasting though it decays over time as individuals accumulate new experiences. Modern brain science helps understand these processes. Evidence on neural plasticity reveals that personal experiences and learning alter the strength of neural connections and fine-tune the brain structure to those past experiences (“use-dependent brain”). I show that experience effects help understand belief formation and decision-making in a wide range of economic applications, including inflation, home purchases, mortgage choices, and consumption expenditures. I argue that experience-based learning is broadly applicable to economic decision-making and discuss topics for future research in education, health, race, and gender economics.
Article
Retinoic acid, the active metabolite of vitamin A, is important for vertebrate cognition and hippocampal plasticity, but few studies have examined its role in invertebrate learning and memory, and its actions in the invertebrate CNS are currently unknown. Using the mollusc Lymnaea stagnalis, we examined operant conditioning of the respiratory behaviour, controlled by a well-defined central pattern generator (CPG), and used citral to inhibit retinoic acid signaling. Both citral and vehicle-treated animals showed normal learning, but citral-treated animals failed to exhibit long-term memory at 24 hrs. Cohorts of citral or vehicle-treated animals were dissected into semi-intact preparations, either 1 hr after training, or after the memory-test 24 hrs later. Simultaneous electrophysiological recordings from the CPG pacemaker cell (RPeD1) and an identified motorneuron (VI) were made whilst monitoring respiratory activity (pneumostome opening). Activity of the CPG pneumostome opener interneuron (IP3) was also monitored indirectly. Vehicle-treated conditioned preparations showed significant changes in network parameters immediately after learning, such as reduced motorneuron bursting activity (from IP3 input), delayed pneumostome opening, and a decoupling of coincident IP3 input within the network. However, citral-treated preparations failed to exhibit these network changes and more closely resembled naïve preparations. Importantly, these citral-induced differences were manifested immediately after training and before any overt changes in the behavioural response (memory impairment). These studies shed light on where and when retinoid signaling might affect a central pattern generating network to promote memory formation during conditioning of a homeostatic behaviour.
Article
Significance Anxiety disorders are among the most prevalent mental illnesses worldwide. Despite significant advances in their treatment, many patients remain treatment resistant. Thus, new treatment modalities and targets are much needed. Therefore, we developed a deep brain stimulation therapy that targets a recently identified anxiety center in the lateral hypothalamus. We show that this therapy rapidly silences anxiety-implicated neurons and immediately relieves diverse anxiety symptoms in a variety of stressful situations. This therapeutic effect occurs without acute or chronic side effects that are typical of many existing treatments, such as physical sedation or memory deficits. These findings identify a clinically applicable new therapeutic strategy for helping patients to manage treatment-resistant anxiety.
Preprint
Full-text available
The neural correlates of decision-making have been investigated extensively, and recent work aims to identify under what conditions cortex is actually necessary for making accurate decisions. We discovered that mice with distinct cognitive experiences, beyond sensory and motor learning, use different cortical areas and neural activity patterns to solve the same task, revealing past learning as a critical determinant of whether cortex is necessary for decision-making. We used optogenetics and calcium imaging to study the necessity and neural activity of multiple cortical areas in mice with different training histories. Posterior parietal cortex and retrosplenial cortex were mostly dispensable for accurate decision-making in mice performing a simple navigation-based decision task. In contrast, these areas were essential for the same simple task when mice were previously trained on complex tasks with delay periods or association switches. Multi-area calcium imaging showed that, in mice with complex-task experience, single-neuron activity had higher selectivity and neuron-neuron correlations were weaker, leading to codes with higher task information. Therefore, past experience sets the landscape for how future tasks are solved by the brain and is a key factor in determining whether cortical areas have a causal role in decision-making.
Article
Full-text available
The lateral hypothalamus (LH) includes several anatomical subregions involved in eating and reward motivation. This study explored localization of function across different LH subregions in controlling food intake stimulated by optogenetic channelrhodopsin excitation, and in supporting laser self-stimulation. We particularly compared the tuberal LH subregion, the posterior LH subregion, and the lateral preoptic area. Local diameters of tissue optogenetically stimulated within the LH were assessed by measuring laser-induced Fos plumes and Jun plumes via immunofluorescence surrounding optic fiber tips. Those plume diameters were used to map localization of function for behavioral effects elicited by LH optogenetic stimulation. Optogenetic stimulation of the tuberal subsection of the LH produced the most robust eating behavior and food intake initially, but produced only mild laser self-stimulation in the same rats. However, after repeated exposures to optogenetic stimulation, tuberal LH behavioral profiles shifted toward more self-stimulation and less food intake. By contrast, stimulation of the lateral preoptic area produced relatively little food intake or self-stimulation, either initially or after extended stimulation experience. Stimulation in the posterior LH subregion supported moderate self-stimulation, but not food intake, and at higher laser intensity shifted valence to evoke escape behaviors. We conclude that the tuberal LH subregion may best mediate stimulation-bound increases in food intake stimulated by optogenetic excitation. However, incentive motivational effects of tuberal LH stimulation may shift toward self-stimulation behavior after repeated stimulation. By contrast, the lateral preoptic area and posterior LH do not as readily elicit either eating behavior or laser self-stimulation, and may be more prone to higher-intensity aversive effects.
Article
Full-text available
Dopamine neurons are proposed to signal the reward prediction error in model-free reinforcement learning algorithms. This term represents the unpredicted or ‘excess’ value of the rewarding event, value that is then added to the intrinsic value of any antecedent cues, contexts or events. To support this proposal, proponents cite evidence that artificially-induced dopamine transients cause lasting changes in behavior. Yet these studies do not generally assess learning under conditions where an endogenous prediction error would occur. Here, to address this, we conducted three experiments where we optogenetically activated dopamine neurons while rats were learning associative relationships, both with and without reward. In each experiment, the antecedent cues failed to acquire value and instead entered into associations with the later events, whether valueless cues or valued rewards. These results show that in learning situations appropriate for the appearance of a prediction error, dopamine transients support associative, rather than model-free, learning. Dopamine neurons are proposed to signal the reward prediction error in model-free reinforcement learning algorithms. Here, the authors show that when given during an associative learning task, optogenetic activation of dopamine neurons causes associative, rather than value, learning.
Article
Full-text available
Theories of functioning in the medial prefrontal cortex are distinct across appetitively and aversively motivated procedures. In the appetitive domain, it is argued that the medial prefrontal cortex is important for producing adaptive behavior when circumstances change. This view advocates a role for this region in using higher-order information to bias performance appropriate to that circumstance. Conversely, literature born out of aversive studies has led to the theory that the prelimbic region of the medial prefrontal cortex is necessary for the expression of conditioned fear, whereas the infralimbic region is necessary for a decrease in responding following extinction. Here, the argument is that these regions are primed to increase or decrease fear responses and that this tendency is gated by subcortical inputs. However, we believe the data from aversive studies can be explained by a supraordinate role for the medial prefrontal cortex in behavioral flexibility, in line with the appetitive literature. Using a dichotomy between the voluntary control of behavior and the execution of well-trained responses, we attempt to reconcile these theories. We argue that the prelimbic region exerts voluntary control over behavior via top-down modulation of stimulus–response pathways according to task demands, contextual cues, and how well a stimulus predicts an outcome. Conversely, the infralimbic region promotes responding based on the strength of stimulus–response pathways determined by experience with reinforced contingencies. This system resolves the tension between executing voluntary actions sensitive to recent changes in contingencies, and responses that reflect the animal’s experience across the long run.
Article
Full-text available
How are decisions made between different goods? One theory spanning several fields of neuroscience proposes that their values are distilled to a single common neural currency, the calculation of which allows for rational decisions. The orbitofrontal cortex (OFC) is thought to play a critical role in this process, based on the presence of neural correlates of economic value in lateral OFC in monkeys and medial OFC in humans. We previously inactivated lateral OFC in rats without affecting economic choice behavior. Here we inactivated medial OFC in the same task, again without effect. Behavior in the same rats was disrupted by inactivation during progressive ratio responding previously shown to depend on medial OFC, demonstrating the efficacy of the inactivation. These results indicate that medial OFC is not necessary for economic choice, bolstering the proposal that classic economic choice is likely mediated by multiple, overlapping neural circuits.
Article
Full-text available
Prediction errors are critical for associative learning. In the brain, these errors are thought to be signaled, in part, by midbrain dopamine neurons. However, while there is substantial direct evidence that brief increases in the firing of these neurons can mimic positive prediction errors, there is less evidence that brief pauses mimic negative errors. While pauses in the firing of midbrain dopamine neurons can substitute for missing negative prediction errors to drive extinction, it has been suggested that this effect might be due to changes in salience rather than the operation of this signal as a negative prediction error. Here we address this concern by showing that the same pattern of inhibition will create a cue able meet the classic definition of a conditioned inhibitor by showing suppression of responding in a summation test and slower learning in a retardation test. Importantly, these classic criteria were designed to rule out explanations founded on attention or salience, thus the results cannot be explained in this manner. We further show that this pattern of behavior is not produced by a single, prolonged, ramped period of inhibition, suggesting that it is precisely timed, sudden change and not duration that conveys the teaching signal.Significance Statement: Here we show that brief pauses in the firing of midbrain dopamine neurons are sufficient to produce a cue that meets the classic criteria defining a conditioned inhibitor, or a cue that predicts the omission of a reward. These criteria were developed to distinguish actual learning from salience or attentional effects, thus these results formally show that brief pauses in the firing of dopamine neurons can serve as key teaching signals in the brain. Interestingly this was not true for gradual prolonged pauses, suggesting it is the dynamic change in firing that serves as the teaching signal.
Book
Research in cognitive psychology, linguistics, and artificial intelligence – the three disciplines that have the most direct application to an understanding of the mental processes in reading – is presented in this multilevel work, originally published in 1980, that attempts to provide a systematic and scientific basis for understanding and building a comprehensive theory of reading comprehension. The major focus is on understanding the processes involved in the comprehension of written text. Underlying most of the contributions is the assumption that skilled reading comprehension requires a coordination of text with context in a way that goes far beyond simply chaining together the meanings of a string of decoded words. The topics discussed are divided into five general areas: Global Issues; Text Structure; Language, Knowledge of the World, and Inference; Effects of Prior Language Experience; and Comprehension Strategies and Facilitators, and represent a broad base of methodology and data that should be of interest not only to those concerned with the reading process, but also to basic science researchers in psychology, linguistics, artificial intelligence, and related disciplines. © 1980 by Lawrence Erlbaum Associates, Inc. All rights reserved.
Article
Over the years Pavlovian fear conditioning has proved to be a powerful model to investigate the neural underpinnings of aversive associative memory formation. Although it is well appreciated that plasticity occurring at excitatory synapses within the basolateral complex of the amygdala (BLA) plays a critical role in associative memory formation, recent evidence suggests that plasticity within the amygdala is more distributed than previously appreciated. In particular, studies demonstrate that plasticity in the central nucleus (CeA) is critical for the acquisition of conditioned fear. In addition, a variety of interneuron populations within the amygdala, defined by unique neurochemical markers, contribute to distinct aspects of stimulus processing and memory formation during fear conditioning. Here, we will review and summarize recent advances in our understanding of amygdala networks and how unique players within this network contribute to synaptic plasticity associated with the acquisition of conditioned fear.
Article
Sensory preconditioning has been used to implicate midbrain dopamine in modelbased learning, contradicting the view that dopamine transients reflect model-free value. However, it has been suggested that model-free value might accrue directly to the preconditioned cue through mediated learning. Here, building on previous work (Sadacca et al., 2016), we address this question by testing whether a preconditioned cue will support conditioned reinforcement in rats. We found that while both directly conditioned and second-order conditioned cues supported robust conditioned reinforcement, a preconditioned cue did not. These data show that the preconditioned cue in our procedure does not directly accrue model-free value and further suggest that the cue may not necessarily access value even indirectly in a model-based manner. If so, then phasic response of dopamine neurons to cues in this setting cannot be described as signaling errors in predicting value.
Article
Phasic dopamine responses are thought to encode a prediction-error signal consistent with model-free reinforcement learning theories. However, a number of recent findings highlight the influence of model-based computations on dopamine responses, and suggest that dopamine prediction errors reflect more dimensions of an expected outcome than scalar reward value. Here, we review a selection of these recent results and discuss the implications and complications of model-based predictions for computational theories of dopamine and learning.