No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Disconnection of basolateral amygdala and insular cortex disrupts conditioned approach in Pavlovian lever autoshaping
Abstract
Previously established individual differences in appetitive approach and devaluation sensitivity observed in goal- and sign-trackers may be attributed to differences in the acquisition, modification, or use of associative information in BLA pathways. Here, we sought to determine the extent to which communication of associative information between BLA and anterior portions of insular cortex (IC) supports ongoing Pavlovian conditioned approach behaviors in sign- and goal-tracking rats, in the absence of manipulations to outcome value. We hypothesized that the BLA mediates goal-, but not sign-, tracking approach through interactions with the IC, a brain region involved in supporting flexible behavior. We first trained rats in Pavlovian lever autoshaping to determine their sign- or goal-tracking tendency. During alternating test sessions, we gave unilateral intracranial injections of vehicle or a cocktail of gamma-aminobutyric acid (GABA) receptor agonists, baclofen and muscimol, unilaterally into the BLA and contralaterally or ipsilaterally into the IC prior to reinforced lever autoshaping sessions. Consistent with our hypothesis we found that contralateral inactivation of BLA and IC increased the latency to approach the food cup and decreased the number of food cup contacts in goal-trackers. While contralateral inactivation of BLA and IC did not affect the total number of lever contacts in sign-trackers, this manipulation increased the latency to approach the lever. Ipsilateral inactivation of BLA and IC did not impact approach behaviors in Pavlovian lever autoshaping. These findings, contrary to our hypothesis, suggest that communication between BLA and IC maintains a representation of initially learned appetitive associations that commonly support the initiation of Pavlovian conditioned approach behavior regardless of whether it is directed at the cue or the location of reward delivery.
... Similarly, the BLA evaluates sensory information (Brown et al., 2013) and adjusts the association between emotional stimulus and outcome value (Nasser et al., 2017;Parkes & Balleine, 2013). The BLA has also been implicated in memory processes for emotional experiences (Ferreira et al., 2005;Yoon et al., 2016). ...
... The BLA is involved in fear learning, by integrating information from the sensory cortex, thalamus and dACC in people with anxiety (Hakamata et al., 2020). The insula could receive signals of discomfort from the body, and then integrate the signals and send the discomfort information to the amygdala (Nasser et al., 2017;Uematsu et al., 2015). Ferreira et al. (2005) found that the connection between the BLA and insula is important during the formation of disgust feelings. ...
Emotional regulation is known to be associated with activity in the amygdala. The amygdala is an emotion-generative region that comprises of structurally and functionally distinct nuclei. However, little is known about the contributions of different frontal-amygdala sub-region pathways to emotion regulation. Here, we investigated how functional couplings between frontal regions and amygdala sub-regions are involved in different spontaneous emotion regulation processes by using an individual-difference approach and a generalized psycho-physiological interaction (gPPI) approach. Specifically, 50 healthy participants reported their dispositional use of spontaneous cognitive reappraisal and expressive suppression in daily life and their actual use of these two strategies during the performance of an emotional-picture watching task. Results showed that functional coupling between the orbitofrontal cortex (OFC) and the basolateral amygdala (BLA) was associated with higher scores of both dispositional and actual uses of reappraisal. Similarly, functional coupling between the dorsolateral prefrontal cortex (dlPFC) and the centromedial amygdala (CMA) was associated with higher scores of both dispositional and actual uses of suppression. Mediation analyses indicated that functional coupling of the right OFC-BLA partially mediated the association between reappraisal and emotional response, irrespective of whether reappraisal was measured by dispositional use (indirect effect(SE)=-0.2021 (0.0811), 95%CI(BC)= [-0.3851, -0.0655]) or actual use (indirect effect(SE)=-0.1951 (0.0796), 95%CI(BC)= [-0.3654, -0.0518])). These findings suggest that spontaneous reappraisal and suppression involve distinct frontal- amygdala functional couplings, and the modulation of BLA activity from OFC may be necessary for changing emotional response during reappraisal.
... Preclinical work suggests largely overlapping neural circuits are involved in Pavlovian learning processes, including OFC [132], dorsolateral PFC [97], nucleus accumbens (NAcc) [133], subthalamic nucleus [134], amygdala [135][136][137][138][139][140], hippocampus [139], and insula [138], which are widely preserved across species [141,142]. Recent work has shown the relevance of adrenergic, cannabinoid receptor, and NMDA signaling for Pavlovian conditioning [99,[143][144][145][146]. ...
... Preclinical work suggests largely overlapping neural circuits are involved in Pavlovian learning processes, including OFC [132], dorsolateral PFC [97], nucleus accumbens (NAcc) [133], subthalamic nucleus [134], amygdala [135][136][137][138][139][140], hippocampus [139], and insula [138], which are widely preserved across species [141,142]. Recent work has shown the relevance of adrenergic, cannabinoid receptor, and NMDA signaling for Pavlovian conditioning [99,[143][144][145][146]. ...
Purpose of Review
Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.
Recent Findings
The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.
Summary
While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.
... Specifically, the nuclei of basolateral amygdala (BLA) and centromedial amygdala (CMA) play a vital role in the input processing of emotional information and the generation of behavioral responses, respectively ( Danilo et al., 2013 ;Joseph, 2000 ). Similarly, the BLA evaluates sensory information ( Brown et al., 2013 ) and adjusts the association between emotional stimulus and outcome value ( Nasser et al., 2017 ;Parkes and Balleine, 2013 ). The BLA has also been implicated in memory processes for emotional experiences ( Ferreira et al., 2005 ;Yoon et al., 2016 ). ...
... The BLA is involved in fear learning, by integrating information from the sensory cortex, thalamus and dACC in people with anxiety ( Hakamata et al., 2020 ). The insula could receive signals of discomfort from the body, and then integrate the signals and send the discomfort information to the amygdala ( Nasser et al., 2017 ;Uematsu et al., 2015 ). Ferreira et al. (2005) found that the connection between the BLA and insula is important during the formation of disgust feelings. ...
... The reality is that extra calories and intoxication both contribute to the reinforcing effects of ingested alcohol. However, the within-session behavior patterns exhibited by the Paired group here and in our previous study (Cofresí et al., 2018) are not observed in studies with fooddeprived rats presented with cues that predict food pellet delivery, where the homeostatic drive for calories likely provides the primary motivation for learning (e.g., Nasser, Lafferty, Lesser, Bacharach, & Calu, 2018). ...
... The only brain region in which we observed noteworthy differential Fos expression between the Paired and Unpaired groups was the insula (Fig. 7A). Activity in the insula appears to be important for maintaining cue-reward associative memory (Nasser et al., 2018). Additionally, deactivation of the insula appears to be important for the interoceptive effects of alcohol (Jaramillo, Randall, Frisbee, & Besheer, 2016). ...
... Another consideration in our projection-specific inactivation experiments is that vHC projections to the MO can collateralize to the insular cortex (IC) (Verwer et al., 1997). We think that these collaterals likely did not grossly affect outcomes here, because the IC is associated with Pavlovian (and not instrumental) conditioning (Kusumoto-Yoshida et al., 2015;Nasser et al., 2018;Parkes et al., 2016), including CTA (Ferreira et al., 2002;Gutié rrez et al., 1999), such that inactivation disrupts avoidance of a LiClpaired food. By contrast, none of our manipulations disrupted CTA, and we did not detect fluorescence in the IC in projection-specific inactivation experiments, suggesting that IC collaterals were sparse and did not grossly affect our experiments ( Figure S6). ...
In everyday life, we mentally represent possible consequences of our behaviors and integrate specific outcome values into existing knowledge to inform decisions. The medial orbitofrontal cortex (MO) is necessary to adapt behaviors when outcomes are not immediately available—when they and their values need to be envisioned. Nevertheless, neurobiological mechanisms remain unclear. We find that the neuroplasticity-associated neurotrophin receptor tropomyosin receptor kinase B (TrkB) is necessary for mice to integrate outcome-specific value information into choice behavior. This function appears attributable to memory updating (and not retrieval) and the stabilization of dendritic spines on excitatory MO neurons, which led us to investigate inputs to the MO. Ventral hippocampal (vHC)-to-MO projections appear conditionally necessary for value updating, involved in long-term aversion-based value memory updating. Furthermore, vHC-MO-mediated control of choice is TrkB dependent. Altogether, we reveal a vHC-MO connection by which specific value memories are updated, and we position TrkB within this functional circuit.
... were differences in houselight illumination cue-related neural activity between the paired and unpaired groups in the anterior insular cortex-an important region for cue-reward associative memory maintenance (Nasser et al., 2018). Importantly, in our study we could not parse out whether the differences we observed in the insula were due to memory formation or memory expression . ...
In this selective review article, we showcase our collaborations with our colleague, Dr. Nadia Chaudhri. Dr. Chaudhri was an esteemed colleague and researcher who contributed greatly to our understanding of Pavlovian alcohol conditioning. From 2014 to 2019, we collaborated with Nadia. Here, we reflect on our friendship, the work we did together, and the continued impact on the field.
... Other studies have also demonstrated that anterior insula is involved in devaluation (Balleine and Dickinson, 2000;Parkes et al., 2015;Pelloux et al., 2013) and taste aversion (Cubero et al., 1999;Ferreira et al., 2005;Kayyal et al., 2019;Miranda and McGaugh, 2004). Recent work from the Calu lab has also found that inactivating the pathway between basolateral amygdala and anterior insula reduced food cup contacts and increased the latency to approach the food cup in rats that had previously been identified as goaltrackers (Nasser et al., 2018). ...
The insula has become a significant brain region in the study of both normal and impaired behavior and decision-making and has emerged as an important contributor to drug addiction. Consistent with this literature, in a previous study, we found that neural signals in rat insula encode anticipation and contextual global reward value during performance of an odor-guided delay/size choice task, and that these signals are disrupted by prior cocaine self-administration. Still, it is unknown if insula is critical for performance of this task under normal circumstances. Here, we sought to elucidate the functional role of these signals by lesioning the same region of anterior insula we previously recorded from. In addition to examining behavior during decision-making, we characterized behavior during autoshaping to further assess insula’s role in behavior. We found insula damage resulted in reduced accuracy and faster reaction times, without affecting rats’ choice of high-value reward, and that insula lesions reduced sign-tracking behavior. These results suggest that insula contributes to our odor-guided delay/size choice task via mechanisms that impact the control that environmental stimuli have on behavior.
... Usually, disconnections studies show that unilateral inactivation or lesions of two brain areas in both hemispheres affects memory, while ipsilateral treatments do not impair memory (Hernandez et al., 2017;Holland, 2007;Keefer & Petrovich, 2020;Nasser, Lafferty, Lesser, Bacharach, & Calu, 2018;Warburton, Baird, Morgan, Muir, & Aggleton, 2001). However, some functional connectivity studies observed impaired memory when transiently inactivating both contralaterally and ipsilaterally (Baker, Rao, Rivera, Garcia, & Mizumori, 2019;Gilmartin, Kwapis, & Helmstetter, 2012;Mathis et al., 2017;Scott et al., 2020). ...
Recognition memory can rely on three components: “what”, “where” and “when”. Recently we demonstrated that the anterior retrosplenial cortex (aRSC), like the perirhinal cortex (PRH) and unlike the hippocampus (HP), is required for consolidation of the “what” component. Here, we aimed at studying which brain structures interact with the aRSC to process object recognition (OR) memory in rats. We studied the interaction of six brain structures that are connected to the aRSC during OR memory processing: PRH, medial prefrontal cortex (mPFC), anteromedial thalamic nuclei (AM), medial entorhinal cortex (MEC), anterior cingulate cortex (ACC) and the dorsal HP (dHP). We previously described the role of the PRH and dHP, so we first studied the participation of the mPFC, AM, MEC and ACC in OR memory consolidation by bilateral microinfusions of the GABAA receptor agonist muscimol. We observed an impairment in OR long-term memory (LTM) when inactivating the mPFC, the AM and the MEC, but not the ACC. Then, we studied the functional connections by unilateral inactivation of the aRSC and each one of the six structures in the same (ipsilateral) or the opposite (contralateral) hemisphere. Our results showed an amnesic LTM effect in rats with ipsilateral inactivations of aRSC-PRH, aRSC-mPFC, aRSC-AM, or aRSC-MEC. On the other hand, we observed memory impairment when aRSC-ACC were inactivated in opposite hemispheres, and no effect when the aRSC-dHP connection was inactivated. Thus, our ipsilateral inactivation findings reveal that the RSC and, at least one brain region required in OR LTM processing are essential to consolidate OR memory. In conclusion, our results show that several cortico-cortical and cortico-thalamic pathways are important for OR memory consolidation.
... Di Pietro et al., 2004;Kesner and Gilbert, 2007), including Pavlovian reward learning (AP+2.8, Nasser et al., 2018). ...
Insula function is considered critical for many motivated behaviors, with proposed functions ranging from attention, behavioral control, emotional regulation, goal-directed and aversion-resistant responding. Further, the insula is implicated in many neuropsychiatric conditions including substance abuse. More recently, multiple insula subregions have been distinguished based on anatomy, connectivity, and functional contributions. Generally, posterior insula is thought to encode more somatosensory inputs, which integrate with limbic/emotional information in middle insula, that in turn integrate with cognitive processes in anterior insula. Together, these regions provide rapid interoceptive information about the current or predicted situation, facilitating autonomic recruitment and quick, flexible action. Here, we seek to create a robust foundation from which to understand potential subregion differences, and provide direction for future studies. We address subregion differences across humans and rodents, so that the latter's mechanistic interventions can best mesh with clinical relevance of human conditions. We first consider the insula's suggested roles in humans, then compare subregional studies, and finally describe rodent work. One primary goal is to encourage precision in describing insula subregions, since imprecision (e.g. including both posterior and anterior studies when describing insula work) does a disservice to a larger understanding of insula contributions. Additionally, we note that specific task details can greatly impact recruitment of various subregions, requiring care and nuance in design and interpretation of studies. Nonetheless, the central ethological importance of the insula makes continued research to uncover mechanistic, mood, and behavioral contributions of paramount importance and interest.
This article is part of the special Issue on ‘Neurocircuitry Modulating Drug and Alcohol Abuse'.
... electrophysiological studies (Ahrens et al., 2016;Stringfield et al., 2017). Within this circuit, it has been demonstrated that sign-tracking is modulated by other neurotransmitter systems (e.g., glutamatergic, cannabinoid, cholinergic, adrenergic, opioidergic, etc.) outside DA signaling (Bacharach et al., 2018;Chow & Beckmann, 2018;DiFeliceantonio and Berridge, 2012;Pasquariello et al., 2018). ...
Cues that are paired with unconditioned, rewarding stimuli can acquire rewarding properties themselves through a process known as the attribution of incentive salience. When previously neutral cues are imbued with incentive salience, they become attractive, “wanted” stimuli capable of motivating behavior. Pavlovian conditioned approach procedures are commonly used to investigate the attribution of incentive salience in rodents. During Pavlovian conditioned approach training, two conditioned responses develop: sign-tracking (behavior directed towards a reward-related cue) and goal-tracking (behavior directed towards the site of reward delivery). Goal-trackers and sign-trackers both use the reward-related cue as a predictor of reward delivery; however, only sign-trackers attribute it with incentive salience and are more vulnerable to addiction-like behaviors, such as cue-induced reinstatement of drug-seeking. Currently, it is known that sign-tracking behavior is dependent on dopamine in the nucleus accumbens, a central hub in the ‘motive circuit,’ an array of mesocorticolimbic brain regions that process incentive stimuli. However, the role of other signaling pathways and the contribution of afferent brain regions within the motive circuit to sign-tracking behavior is poorly understood. In this dissertation, I demonstrate that the ventral hippocampus is a part of the motive circuit, regulating sign-tracking behavior and dopamine signaling in the nucleus accumbens. In addition, I show that the motive circuit can be manipulated environmentally and pharmacologically using prolonged stress and subanesthetic ketamine, respectively, to decrease sign-tracking behavior. Taken together, the results of this dissertation advance our understanding of the functional neurocircuitry of sign-tracking behavior and how it is influenced by environmental and pharmacological manipulations of the motive circuit.
... Lesioning the BLA, inactivating CamKII-containing BLA neurons or inactivating BLA projections to the orbitofrontal cortex or the nucleus accumbens disrupts cue-triggered potentiation of reward seeking (Blundell et al. 2001;Corbit and Balleine 2005;Derman et al. 2020;Lichtenberg et al. 2017;Shiflett and Balleine 2010) (see also Gabriele and See 2010;Puaud et al. 2021). Similarly, pharmacological disconnection of the BLA and insular cortex disrupts cue-evoked conditioned approach behaviours (Nasser et al. 2018). Second, neural activity within the BLA is sufficient to potentiate the incentive motivational properties of reward cues. ...
RationaleReward-associated cues can trigger incentive motivation for reward and invigorate reward-seeking behaviour via Pavlovian-to-instrumental transfer (PIT). Glutamate signaling within the basolateral amygdala (BLA) modulates cue-triggered increases in incentive motivation. However, the role of BLA metabotropic group II glutamate (mGlu2/3) receptors is largely unknown.Objectives
In Experiment 1, we characterized cue-triggered increases in incentive motivation for water reward using the PIT paradigm. In Experiment 2, we assessed the influence of intra-BLA microinjections of the mGlu2/3 receptor agonist LY379268 on this effect.Methods
Water-restricted male Sprague–Dawley rats learned to press a lever for water. Separately, they learned to associate one of two auditory cues with free water. On test days, rats could lever press under extinction conditions (no water), with intermittent, non-contingent CS+ and CS- presentations. In Experiment 1, rats were tested under baseline conditions. In Experiment 2, rats received intra-BLA microinjections of LY379268 (0, 3 and 6 \({\mu }\)g/hemisphere) before testing.ResultsAcross experiments, CS+, but not CS-, presentations increased water-associated lever pressing during testing, even though responding was reinforced neither by water nor the CS+. Intra-BLA LY379268 abolished both CS+ potentiated pressing on the water-associated lever and CS+ evoked conditioned approach to the site of water delivery. LY379268 did not influence locomotion or instrumental and Pavlovian response rates during intervals between CS presentations or during the CS-, indicating no motor effects.ConclusionsmGlu2/3 receptor activity in the BLA mediates cue-triggered potentiation of incentive motivation for reward, suppressing both cue-induced increases in instrumental pursuit of the reward and anticipatory approach behaviour.
... Lesioning the BLA, inactivating CamKIIcontaining BLA neurons or inactivating BLA projections to the orbitofrontal cortex or the nucleus accumbens disrupt cue-triggered potentiation of reward seeking (Lichtenberg et al., 2017;Corbit & Balleine, 2005;Blundell et al., 2001;Derman et al., 2020;Shiflett & Balleine, 2010, see also Gabriele & See, 2010;Puaud et al., 2021). Similarly, pharmacological disconnection of the BLA and insular cortex disrupts CS-evoked conditioned approach behaviours (Nasser et al., 2018). Second, neural activity within the BLA is sufficient to potentiate the incentive motivational properties of reward cues. ...
Rationale
Reward-associated cues can trigger incentive motivation for reward and invigorate reward-seeking behaviour via Pavlovian-to-instrumental transfer (PIT). Glutamate signaling within the basolateral amygdala (BLA) modulates cue-triggered increases in incentive motivation. However, the role of BLA metabotropic group II glutamate (mGlu 2/3 ) receptors is largely unknown.
Objectives
In Experiment 1, we characterized cue-triggered increases in incentive motivation for water reward using the PIT paradigm. In Experiment 2, we assessed the influence of intra-BLA microinjections of the mGlu 2/3 receptor agonist LY379268 on this effect.
Methods
Water-restricted male Sprague-Dawley rats learned to press a lever for water. Separately, they learned to associate one of two auditory cues with free water. On test days, rats could lever press under extinction conditions (no water), with intermittent, non-contingent CS+ and CS- presentations. In Experiment 1, rats were tested under baseline conditions. In Experiment 2, rats received intra-BLA microinjections of LY379268 (0, 3 and 6 μg/hemisphere) before testing.
Results
Across experiments, CS+, but not CS- presentations increased water-associated lever pressing during testing, even though responding was reinforced neither by water nor the CS+. Intra-BLA LY379268 abolished both CS+ potentiated pressing on the water-associated lever and CS+ evoked conditioned approach to the site of water delivery. LY379268 did not influence locomotion or instrumental and Pavlovian response rates during intervals between CS presentations or during the CS-, indicating no motor effects.
Conclusions
mGlu 2/3 receptor activity in the BLA mediates CS-triggered potentiation of incentive motivation for reward, suppressing both CS-induced increases in instrumental pursuit of the reward and anticipatory approach behaviour.
... (No rats shifted from sign tracking to goal tracking in their study.) In addition to prior learning history, neurological damage and drugs affect tendencies to track signs rather than goals (Byrom & Murphy, 2018;Doremus-Fitzwater & Spear, 2011;Nasser, Lafferty, Lesser, Bacharach, & Calu, 2018;Naeem & White, 2016; but see Garcia-Fuster, Parsegian, Watson, Akil, & Flagel, 2017) and is reportedly linked to dopaminergic activity at different receptor sites (Flagel, Watson, Robinson, & Akil, 2007). It makes sense from an evolutionary perspective that sign tracking would be critical for autoshaping. ...
Stimulus–stimulus pairing (SSP) is a procedure used by behavior analysis practitioners that capitalizes on respondent conditioning processes to elicit vocalizations. These procedures usually are implemented only after other, more customary methods (e.g., standard echoic training via modeling) have been exhausted. Unfortunately, SSP itself has mixed research support, probably because certain as-yet-unidentified procedural variations are more effective than others. Even when SSP produces (or increases) vocalizations, its effects can be short-lived. Although specific features of SSP differ across published accounts, fundamental characteristics include presentation of a vocal stimulus proximal with presentation of a preferred item. In the present article, we draw parallels between SSP procedures and autoshaping, review factors shown to affect autoshaping, and interpret autoshaping research for suggested SSP tests and applications. We then call for extended use and reporting of SSP in behavior-analytic treatments. Finally, three bridges created by this article are identified: basic-applied, respondent–operant, and behavior analysis with other sciences.
... By this logic, 878 we predict that neurons in vmPFC should exhibit stronger correlations with licking CRs than 879 neurons in OFC. Apart from the VS and vmPFC (which are specifically implicated in oromotor 880 responses) other potential regions of interest include the amygdala and insular cortex, owing to 881 their involvement in other forms of conditioned appetitive behavior such as autoshaping 882 (Cardinal et al., 2002;Nasser et al., 2018), and the anterior cingulate cortex, owing to its role in 883 motivation more generally (Cohen et al., 1999;Darby et al., 2018). 884 ...
Neural representations of value underlie many behaviors that are crucial for survival. Previously, we found that value representations in primate orbitofrontal cortex (OFC) are modulated by attention, specifically, by overt shifts of gaze toward or away from reward-associated visual cues (McGinty et al., 2016). Here, we investigate the influence of overt attention on behavior by asking how gaze shifts correlate with reward anticipatory responses and whether activity in OFC mediates this correlation. Macaque monkeys viewed pavlovian conditioned appetitive cues on a visual display, while the fraction of time they spent looking toward or away from the cues was measured using an eye tracker. Also measured during cue presentation were the reward anticipation, indicated by conditioned licking responses (CRs), and single-neuron activity in OFC. In general, gaze allocation predicted subsequent licking responses: the longer the monkeys spent looking at a cue at a given time point in a trial, the more likely they were to produce an anticipatory CR later in that trial, as if the subjective value of the cue were increased. To address neural mechanisms, mediation analysis measured the extent to which the gaze-CR correlation could be statistically explained by the concurrently recorded firing of OFC neurons. The resulting mediation effects were indistinguishable from chance. Therefore, while overt attention may increase the subjective value of reward-associated cues (as revealed by anticipatory behaviors), the underlying mechanism remains unknown, as does the functional significance of gaze-driven modulation of OFC value signals.
... Additionally, blood oxygen level-dependent fMRI signals are increased in the insular of cocaine and methamphetamine addicts when exposed to drug-associated cues (Garavan et al., 2000;Yin et al., 2012). Finally, damage to the insular cortex reduces relapse rates in cocaine addicts (Naqvi et al., 2007;Gaznick et al., 2014). These data, combined with our preclinical studies, highlight a pivotal role for the AI in relapse. ...
Humans with alcohol use disorder typically abstain because of the negative consequences associated with excessive drinking, and exposure to contexts previously associated with alcohol use can trigger relapse. We used a rat model that captures a characteristic of this human condition: namely voluntary abstinence from alcohol use because of contingent punishment. There is substantial variability in the propensity to relapse following extended periods of abstinence, and this is a critical feature preventing the successful treatment of alcohol use disorder. Here we examined relapse following acute or prolonged abstinence. In male alcohol preferring P rats, we found an increased propensity to relapse in Context B, the punishment context after prolonged abstinence. Next, we found that neither alcohol intake history nor the motivational strength of alcohol predicted the propensity to relapse. We next examined the putative circuitry of context-induced relapse to alcohol seeking following prolonged abstinence using Fos as a marker of neuronal activation. The anterior insular cortex (AI) was the only brain region examined where Fos expression correlated with alcohol seeking behavior in Context B after prolonged abstinence. Finally, we used local infusion of GABAA and GABAB receptor agonists (muscimol + baclofen) to show a causal role of the AI in context-induced relapse in Context B, the punishment context after prolonged abstinence. Our results show that there is substantial individual variability in the propensity to relapse in the punishment-associated context after prolonged abstinence, and this is mediated by activity in the AI.SIGNIFICANCE STATEMENT A key feature of alcohol use disorder is that sufferers show an enduring propensity to relapse throughout their lifetime. Relapse typically occurs despite the knowledge of adverse consequences including health complications or relationship breakdowns. Here we use a recently developed rodent model that recapitulates this behavior. After an extended period of abstinence, relapse propensity is markedly increased in the "adverse consequence" environment, akin to humans with alcohol use disorder relapsing in the face of adversity. From a circuitry perspective, we demonstrate a causal role of the anterior insular cortex in relapse to alcohol seeking after extended abstinence following punishment imposed voluntary cessation of alcohol use.
... Usually, disconnections studies show that unilateral inactivation or lesions of two brain areas in both hemispheres affects memory, while ipsilateral treatments do not impair memory (Hernandez et al., 2017;Holland, 2007;Keefer & Petrovich, 2020;Nasser, Lafferty, Lesser, Bacharach, & Calu, 2018;Warburton, Baird, Morgan, Muir, & Aggleton, 2001). However, some functional connectivity studies observed impaired memory when transiently inactivating both contralaterally and ipsilaterally (Baker, Rao, Rivera, Garcia, & Mizumori, 2019;Gilmartin, Kwapis, & Helmstetter, 2012;Mathis et al., 2017;Scott et al., 2020). ...
For several decades, neuroscientists have provided many clues that point out the involvement of de novo gene expression during the formation of long-lasting forms of memory. However, information regarding the transcriptional response networks involved in memory formation has been scarce and fragmented. With the advent of genome-based technologies, combined with more classical approaches (i.e., pharmacology and biochemistry), it is now feasible to address those relevant questions--which gene products are modulated, and when that processes are necessary for the proper storage of memories--with unprecedented resolution and scale. Using one-trial inhibitory (passive) avoidance training of rats, one of the most studied tasks so far, we found two time windows of sensitivity to transcriptional and translational inhibitors infused into the hippocampus: around the time of training and 3-6 h after training. Remarkably, these periods perfectly overlap with the involvement of hippocampal cAMP/PKA (protein kinase A) signaling pathways in memory consolidation. Given the complexity of transcriptional responses in the brain, particularly those related to processing of behavioral information, it was clearly necessary to address this issue with a multi-variable, parallel-oriented approach. We used cDNA arrays to screen for candidate inhibitory avoidance learning-related genes and analyze the dynamic pattern of gene expression that emerges during memory consolidation. These include genes involved in intracellular kinase networks, synaptic function, DNA-binding and chromatin modification, transcriptional activation and repression, translation, membrane receptors, and oncogenes, among others. Our findings suggest that differential and orchestrated hippocampal gene expression is necessary in both early and late periods of long-term memory consolidation. Additionally, this kind of studies may lead to the identification and characterization of genes that are relevant for the pathogenesis of complex psychiatric disorders involving learning and memory impairments, and may allow the development of new methods for the diagnosis and treatment of these diseases.
Goal-tracking (GT) rats are sensitive to Pavlovian outcome devaluation while sign-tracking (ST) rats are devaluation insensitive. During outcome devaluation, GT rats flexibly modify responding to cues based on the current value of the associated outcome. However, ST rats rigidly respond to cues regardless of the current outcome value. Prior work demonstrated disconnection of the basolateral amygdala (BLA) and anterior insular cortex (aIC) decreased both GT and ST behaviors. Given the role of these regions in appetitive motivation and behavioral flexibility, we predicted that disrupting BLA to aIC pathway during outcome devaluation would reduce flexibility in GT rats and reduce rigid appetitive motivation in ST rats. We inhibited the BLA to aIC pathway by infusing inhibitory DREADDs (hM4Di-mcherry) or control (mCherry) virus into the BLA and implanted cannulae into the aIC to inhibit BLA terminals using intracranial injections of clozapine N-oxide (CNO). After training, we used a within-subject satiety-induced outcome devaluation procedure in which we sated rats on training pellets (devalued condition) or homecage chow (valued condition). All rats received bilateral CNO infusions into the aIC before brief nonreinforced test sessions. Contrary to our hypothesis, BLA-IC inhibition did not interfere with devaluation sensitivity in GT rats but did make ST behaviors sensitive to devaluation. Intermediate rats showed the opposite effect, showing rigid responding to cues with BLA-aIC pathway inactivation. Together, these results demonstrate BLA-IC projections mediate tracking-specific Pavlovian devaluation sensitivity and highlights the importance of considering individual differences in Pavlovian approach when evaluating circuitry contributions to behavioral flexibility.
Goal-tracking rats are sensitive to Pavlovian outcome devaluation while sign-tracking rats are devaluation insensitive. During outcome devaluation, goal-tracking (GT) rats flexibly modify responding to cues based on the current value of the associated outcome. However, sign-tracking (ST) rats rigidly respond to cues regardless of the current outcome value. Our prior study demonstrated disconnection of the basolateral amygdala (BLA) and anterior insular cortex (aIC) decreased both goal- and sign-tracking behaviors. Given the role of these regions in appetitive motivation and behavioral flexibility we predicted that disrupting BLA to aIC pathway during outcome devaluation would reduce flexibility in GT rats and reduce rigid appetitive motivation in ST rats. We inhibited the BLA to aIC pathway by infusing inhibitory DREADDs (hM4Di-mcherry) or control (mCherry) virus into the BLA and implanted cannulae into the aIC to inhibit BLA terminals using intracranial injections of clozapine N-oxide (CNO). After training, we used a within subject satiety-induced outcome devaluation procedure in which we sated rats on training pellets (devalued condition) or homecage chow (valued condition). All rats received bilateral CNO infusions into the aIC prior to brief non-reinforced test sessions. Contrary to our hypothesis, BLA-IC inhibition did not interfere with devaluation sensitivity in GT rats but did make ST behaviors sensitive to devaluation. Intermediate rats showed the opposite effect, showing rigid in responding to cues with BLA-aIC pathway inactivation. Together, these results demonstrate BLA-IC projections mediate tracking-specific Pavlovian devaluation sensitivity and highlights the importance of considering individual differences in Pavlovian approach when evaluating circuitry contributions to behavioral flexibility.
This review highlights literature relating the anatomy, physiology, and behavioral contributions by projections between rodent prefrontal cortical areas and the basolateral amygdala. These projections are robustly modulated by both environmental experience and exposure to drugs of abuse including ethanol. Recent literature relating optogenetic and chemogenetic dissection of these circuits within behavior both compliments and occasionally challenges roles defined by more traditional pharmacological or lesion-based approaches. In particular, cortico-amygdala circuits help control both aversive and reward-seeking. Exposure to pathology-producing environments or abused drugs dysregulates the relative ‘balance’ of these outcomes. Modern circuit-based approaches have also shown that overlapping populations of neurons within a given brain region frequently govern both aversion and reward-seeking. In addition, these circuits often dramatically influence ‘local’ cortical or basolateral amygdala excitatory or inhibitory circuits. Our understanding of these neurobiological processes, particularly in relation to ethanol research, has just begun and represents a significant opportunity.
This article is part of the special Issue on ‘Neurocircuitry Modulating Drug and Alcohol Abuse'.
Flexible and efficient decision-making in complex environments can be achieved through constant interactions between the goal-directed and habitual systems. While goal-directed behavior is considered dependent upon Response-Outcome (R-O) associations, habits instead rely on Stimulus-Response (S-R) associations. However, the stimuli that support the S-R association underlying habitual responding in typical instrumental procedures are poorly defined. To resolve this issue, we designed a discrete-trials procedure, in which rats must wait for lever insertion and complete a sequence of five lever presses to obtain a reward (20% sucrose or grain-based pellets). Lever insertion thus constituted an audio-visual stimulus signaling the opportunity for reward. Using sensory-specific satiety-induced devaluation, we found that rats trained with grain-based pellets remained sensitive to outcome devaluation over the course of training with this procedure whereas rats trained with a solution of 20% sucrose rapidly developed habit, and that insensitivity to outcome devaluation in rats trained with sucrose did not result from a bias in general satiety. Importantly, although rats trained with pellets were sensitive to satiety-induced devaluation, their performance was not affected by degradation of instrumental contingency and devaluation by conditioned taste aversion (CTA), suggesting that these rats may also have developed habitual responding. To test whether the discrete-trials procedure biases subjects towards habitual responding, we compared discrete-trials to free-running instrumental responding, and found that rats trained with sucrose in a fixed-ratio 5 (FR5) procedure with continuous presentation of the lever were goal-directed. Together, these results demonstrate that discrete presentations of a stimulus predictive of reward availability promoted the formation of S-R habit in rats trained with liquid sucrose. Further research is necessary to explain inconsistencies in sensitivity to outcome devaluation when rats are trained with grain-based pellets.
To make an appropriate decision, one must anticipate potential future rewarding events, even when they are not readily observable. These expectations are generated by using observable information (e.g., stimuli or available actions) to retrieve often quite detailed memories of available rewards. The basolateral amygdala (BLA) and orbitofrontal cortex (OFC) are two reciprocally connected key nodes in the circuitry supporting such outcome-guided behaviors. But there is much unknown about the contribution of this circuit to decision making, and almost nothing known about the whether any contribution is via direct, monosynaptic projections, or the direction of information transfer. Therefore, here we used designer receptor-mediated inactivation of OFC¡BLA or BLA¡OFC projections to evaluate their respective contributions to outcome-guided behaviors in rats. Inactivation of BLA terminals in the OFC, but not OFC terminals in the BLA, disrupted the selective motivating influence of cue-triggered reward representations over reward-seeking decisions as assayed by Pavlovian-to-instrumental transfer. BLA¡OFC projections were also required when a cued reward representation was used to modify Pavlovian conditional goal-approach responses according to the reward’s current value. These projections were not necessary when actions were guided by reward expectations generated based on learned action-reward contingencies, or when rewards themselves, rather than stored memories, directed action. These data demonstrate that BLA¡OFC projections enable the cue-triggered reward expectations that can motivate the execution of specific action plans and allow adaptive conditional responding.
Some rats [sign-trackers (STs)] are prone to attribute incentive salience to reward cues, which can manifest as a propensity to approach and contact pavlovian cues, and for addiction-like behavior. STs also exhibit poor attentional performance, relative to goal-trackers (GTs), which is associated with attenuated acetylcholine (ACh) levels in prefrontal cortex (Paolone et al., 2013). Here, we demonstrate a cellular mechanism, linked to ACh synthesis, that accounts for attenuated cholinergic capacity in STs. First, we found that electrical stimulation of the basal forebrain increased cortical choline transporter (CHT)-mediated choline transport in GTs, paralleled by a redistribution of CHTs to the synaptic plasma membrane. Neither increases in choline uptake nor translocation of CHTs occurred in STs. Second, and consistent with uptake/translocation alterations, STs demonstrated a reduced ability to support cortical ACh release in vivo compared with GTs after reverse-dialysis to elevate extracellular potassium levels. Third, rats were significantly more likely to develop sign-tracking behavior if treated systemically before pavlovian conditioned approach training with the CHT inhibitor VU 6001221. Consistent with its proposed mechanisms, administration of VU6001221 attenuated potassium-evoked ACh levels in prefrontal cortex measured with in vivo microdialysis. We propose that loss of CHT-dependent activation of cortical cholinergic activity in STs degrades top-down executive control over behavior, producing a bias for bottom-up or stimulus-driven attention. Such an attentional bias contributes to nonadaptive reward processing and thus identifies a novel mechanism that can support psychopathology, including addiction.
Significance statement:
Dysfunctional interactions between orbitofrontal cortex (OFC) and the amygdala underlie several mental health disorders, often related to value-based decision making. Understanding the underlying neural circuitry may help to develop therapies for those suffering from mood and anxiety disorders and provide insight into addiction. Here we investigated whether the amygdala must interact with OFC to make adaptive choices. Monkeys learned to perform two different actions: 'tap' for one kind of food reward, 'hold' for another. Then one of the two foods was devalued temporarily. Intact monkeys shifted their choice to whichever action produced the higher-value food; monkeys with crossed surgical disconnection of OFC and the amygdala did not. Thus, OFC and the amygdala must functionally interact to mediate adaptive choices.
Laboratory rats can exhibit marked, qualitative individual differences in the form of acquired behaviors. For example, when exposed to a signal-reinforcer relationship some rats show marked and consistent changes in sign-tracking (interacting with the signal; e.g., a lever) and others show marked and consistent changes in goal-tracking (interacting with the location of the predicted reinforcer; e.g., the food well). Here, stable individual differences in rats’ sign-tracking and goal-tracking emerged over the course of training, but these differences did not generalize across different signal-reinforcer relationships (Experiment 1). This selectivity suggests that individual differences in sign- and goal-tracking reflect differences in the value placed on individual reinforcers. Two findings provide direct support for this interpretation: the palatability of a reinforcer (as measured by an analysis of lick-cluster size) was positively correlated with goal-tracking (and negatively correlated with sign-tracking); and sating rats with a reinforcer affected goal-tracking but not sign-tracking (Experiment 2). These results indicate that the observed individual differences in sign- and goal-tracking behavior arise from the interaction between the palatability or value of the reinforcer and processes of association as opposed to dispositional differences (e.g., in sensory processes, “temperament,” or response repertoire).
Unlabelled:
There is considerable individual variation in the extent to which reward cues are attributed with incentive salience. For example, a food-predictive conditioned stimulus (CS; an illuminated lever) becomes attractive, eliciting approach toward it only in some rats ("sign trackers," STs), whereas others ("goal trackers," GTs) approach the food cup during the CS period. The purpose of this study was to determine how individual differences in Pavlovian approach responses are represented in neural firing patterns in the major output structure of the mesolimbic system, the ventral pallidum (VP). Single-unit in vivo electrophysiology was used to record neural activity in the caudal VP during the performance of ST and GT conditioned responses. All rats showed neural responses to both cue onset and reward delivery but, during the CS period, STs showed greater neural activity than GTs both in terms of the percentage of responsive neurons and the magnitude of the change in neural activity. Furthermore, neural activity was positively correlated with the degree of attraction to the cue. Given that the CS had equal predictive value in STs and GTs, we conclude that neural activity in the VP largely reflects the degree to which the CS was attributed with incentive salience.
Significance statement:
Cues associated with reward can acquire motivational properties (i.e., incentive salience) that cause them to have a powerful influence on desire and motivated behavior. There are individual differences in sensitivity to reward-paired cues, with some individuals attaching greater motivational value to cues than others. Here, we investigated the neural activity associated with these individual differences in incentive salience. We found that cue-evoked neural firing in the ventral pallidum (VP) reflected the strength of incentive motivation, with the greatest neural responses occurring in individuals that demonstrated the strongest attraction to the cue. This suggests that the VP plays an important role in the process by which cues gain control over motivation and behavior.
In an autoshaping task, a single conditioned stimulus (CS; lever insertion) was repeatedly followed by the delivery of an unconditioned stimulus (US; food pellet into an adjacent food magazine) irrespective of the rats' behavior. After repeated training trials, some rats responded to the onset of the CS by approaching and pressing the lever (sign-trackers). Lesions of dorsolateral striatum almost completely eliminated responding to the lever CS while facilitating responding to the food magazine (US). Lesions of the dorsomedial striatum attenuated but did not eliminate responding to the lever CS. Lesions of the basolateral or central nucleus of the amygdala had no significant effects on sign-tracking, but combined lesions of the 2 structures impaired sign-tracking by significantly increasing latency to the first lever press without affecting the number of lever presses. Lesions of the dorsal hippocampus had no effect on any of the behavioral measures. The findings suggest that sign-tracking with a single lever insertion as the CS may consist of 2 separate behaviors learned in parallel: An amygdala-mediated conditioned orienting and approach response and a dorsal striatummediated instrumental response.
Rationale
Some individuals are particularly responsive to reward-associated stimuli (“cues”), including the effects of these cues on craving and relapse to drug-seeking behavior. In the cases of nicotine and alcohol, cues may acquire these abilities via the incentive-enhancing properties of the drug.
Objectives
To determine the interaction between cue-responsivity and nicotine reinforcement, we studied the patterns of nicotine self-administration in rats categorized based on their tendency to approach a food-predictive cue (“sign-trackers”) or a reward-delivery location (“goal-trackers”). In a second experiment, we determined whether nicotine and ethanol altered the incentive value of a food cue.
Methods
Rats were classified as sign- or goal-trackers during a Pavlovian conditioned approach paradigm. Rats then self-administered intravenous nicotine (0.03 mg/kg infusions) followed by extinction and cue-induced reinstatement tests. We also tested the effects of nicotine (0.4 mg/kg base s.c.) or ethanol (0.7 g/kg i.p.) on the approach to, and reinforcing efficacy of, a food cue.
Results
Sign-trackers showed greater reinstatement in response to a nicotine cue. Further, nicotine enhanced sign-tracking but not goal-tracking to a food cue and also enhanced responding for the food cue during the conditioned reinforcement test. Conversely, ethanol reduced sign-tracking and increased goal-tracking, but had no effect on conditioned reinforcement.
Conclusions
Our studies demonstrate that the tendency to attribute incentive value to a food cue predicts enhanced cue-induced reinstatement. Additionally, the incentive value of food cues is differentially modulated by nicotine and ethanol, which may be related to the reinforcing effects of these drugs.
Appetitive sign-tracking, in which reward-paired cues elicit approach that can result in cue interaction, demonstrates how cues acquire motivational value. For example, rats will approach and subsequently interact with a lever insertion cue that signals food delivery upon its retraction. However, lever deflections are rapidly reduced once rats are trained on an omission schedule in which lever interactions cancel food delivery. Here we evaluated the change in sign-tracking response topography in rats exposed to such an omission procedure. Lever deflections dropped precipitously when they canceled reward. However, rats that were on an omission schedule continued to approach, sniff, and contact the lever without pressing it, and did so at comparable rates to rats that were not under an omission schedule. Thus, sign-tracking was maintained, albeit in a different manner, following omission. Such findings show that the motivational attraction to reward cues can be expressed with remarkable persistence and flexibility.
During Pavlovian conditioning, a conditioned stimulus (CS) may act as a predictor of a reward to be delivered in another location. Individuals vary widely in their propensity to engage with the CS (sign tracking) or with the site of eventual reward (goal tracking). It is often assumed that sign tracking involves the association of the CS with the motivational value of the reward, resulting in the CS acquiring incentive value independent of the outcome. However, experimental evidence for this assumption is lacking. In order to test the hypothesis that sign tracking behavior does not rely on a neural representation of the outcome, we employed a reward devaluation procedure. We trained rats on a classic Pavlovian paradigm in which a lever CS was paired with a sucrose reward, then devalued the reward by pairing sucrose with illness in the absence of the CS. We found that sign tracking behavior was enhanced, rather than diminished, following reward devaluation; thus, sign tracking is clearly independent of a representation of the outcome. In contrast, goal tracking behavior was decreased by reward devaluation. Furthermore, when we divided rats into those with high propensity to engage with the lever (sign trackers) and low propensity to engage with the lever (goal trackers), we found that nearly all of the effects of devaluation could be attributed to the goal trackers. These results show that sign tracking and goal tracking behavior may be the output of different associative structures in the brain, providing insight into the mechanisms by which reward-associated stimuli—such as drug cues—come to exert control over behavior in some individuals.
Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS−) and second-order (SOCS+, SOCS−) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.
Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.
Sign- and goal-tracking are differentially associated with drug abuse-related behavior. Recently, it has been hypothesized that sign- and goal-tracking behavior are mediated by different neurobehavioral valuation systems, including differential incentive salience attribution. Herein, we used different conditioned stimuli to preferentially elicit different response types to study the different incentive valuation characteristics of stimuli associated with sign- and goal-tracking within individuals. The results demonstrate that all stimuli used were equally effective conditioned stimuli; however, only a lever stimulus associated with sign- tracking behavior served as a robust conditioned reinforcer and was preferred over a tone associated with goal-tracking. Moreover, the incentive value attributed to the lever stimulus was capable of promoting suboptimal choice, leading to a significant reduction in reinforcers (food) earned. Furthermore, sign- tracking to a lever was more persistent than goal-tracking to a tone under omission and extinction contingencies. Finally, a conditional discrimination procedure demonstrated that sign- tracking to a lever and goal-tracking to a tone were dependent on learned stimulus-reinforcer relations. Collectively, these results suggest that the different neurobehavioral valuation processes proposed to govern sign- and goal-tracking behavior are independent but parallel processes within individuals. Examining these systems within individuals will provide a better understanding of how one system comes to dominate stimulus-reward learning, thus leading to the differential role these systems play in abuse-related behavior.
In natural conditions, gustatory stimuli are typically expected. Anticipatory and contextual cues provide information that allows animals to predict the availability and the identity of the substance to be ingested. Recording in alert rats trained to self-administer tastants following a go signal revealed that neurons in the primary gustatory cortex (GC) can respond to anticipatory cues. These experiments were optimized to demonstrate that even the most general form of expectation can activate neurons in GC, and did not provide indications on whether cues predicting different tastants could be encoded selectively by GC neurons. Here we recorded single-neuron activity in GC of rats engaged in a task where one auditory cue predicted sucrose, while another predicted quinine. We found that GC neurons respond differentially to the two cues. Cue-selective responses develop in parallel with learning. Comparison between cue and sucrose responses revealed that cues could trigger the activation of anticipatory representations. Additional experiments showed that an expectation of sucrose leads a subset of neurons to produce sucrose-like responses even when the tastant was omitted. Altogether, the data show that primary sensory cortices can encode for cues predicting different outcomes, and that specific expectations result in the activation of anticipatory representations.
Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself - a lever - more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.
Drug-associated cues can acquire powerful motivational control over the behavior of addicts, and can contribute to relapse via multiple, dissociable mechanisms. Most preclinical models of relapse focus on only one of these mechanisms: the ability of drug cues to reinforce drug-seeking actions following a period of extinction training. However, in addicts, drug cues typically do not follow seeking actions; they precede them. They often produce relapse by evoking a conditioned motivational state ("wanting" or "craving") that instigates and/or invigorates drug-seeking behavior. Here we used a conflict-based relapse model to ask whether individual variation in the propensity to attribute incentive salience to reward cues predicts variation in the ability of a cocaine cue to produce conditioned motivation (craving) for cocaine. Following self-administration training, responding was curtailed by requiring rats to cross an electrified floor to take cocaine. The subsequent response-independent presentation of a cocaine-associated cue was sufficient to reinstate drug-seeking behavior, despite the continued presence of the adverse consequence. Importantly, there were large individual differences in the motivational properties of the cocaine cue, which were predicted by variation in the propensity to attribute incentive salience to a food cue. Finally, a dopamine antagonist injected into the nucleus accumbens core attenuated, and amphetamine facilitated, cue-evoked cocaine seeking, implicating dopamine signaling in cocaine cue-evoked craving. These data provide a promising preclinical approach for studying sources of individual variation in susceptibility to relapse due to conditioned craving and implicate mesolimbic dopamine in this process.
Choice between goal-directed actions is determined by the relative value of their consequences. Such values are encoded during incentive learning and later retrieved to guide performance. Although the basolateral amygdala (BLA) and the gustatory region of insular cortex (IC) have been implicated in these processes, their relative contribution is still a matter of debate. Here we assessed whether these structures interact during incentive learning and retrieval to guide choice. In these experiments, rats were trained on two actions for distinct outcomes after which one of the two outcomes was devalued by specific satiety immediately before a choice extinction test. We first confirmed that, relative to appropriate controls, outcome devaluation recruited both the BLA and IC based on activation of the immediate early gene Arc; however, we found that infusion of the NMDAr antagonist ifenprodil into the BLA only abolished outcome devaluation when given before devaluation. In contrast, ifenprodil infusion into the IC was effective whether made before devaluation or test. We hypothesized that the BLA encodes and the IC retrieves incentive value for choice and, to test this, developed a novel sequential disconnection procedure. Blocking NMDAr activation unilaterally in the BLA before devaluation and then contralaterally in the IC before test abolished selective devaluation. In contrast, reversing the order of these infusions left devaluation intact. These results confirm that the BLA and IC form a circuit mediating the encoding and retrieval of outcome values, with the BLA encoding and the IC retrieving such values to guide choice.
An inability to adjust choice preferences in response to changes in reward value may underlie key symptoms of many psychiatric disorders, including chemical and behavioral addictions. We developed the rat gambling task (rGT) to investigate the neurobiology underlying complex decision-making processes. As in the Iowa Gambling task, the optimal strategy is to avoid choosing larger, riskier rewards and to instead favor options associated with smaller rewards but less loss and, ultimately, greater long-term gain. Given the demonstrated importance of the orbitofrontal cortex (OFC) and basolateral amygdala (BLA) in acquisition of the rGT and Iowa Gambling task, we used a contralateral disconnection lesion procedure to assess whether functional connectivity between these regions is necessary for optimal decision-making. Disrupting the OFC-BLA pathway retarded acquisition of the rGT. Devaluing the reinforcer by inducing sensory-specific satiety altered decision-making in control groups. In contrast, disconnected rats did not update their choice preference following reward devaluation, either when the devalued reward was still delivered or when animals needed to rely on stored representations of reward value (i.e., during extinction). However, all rats exhibited decreased premature responding and slower response latencies after satiety manipulations. Hence, disconnecting the OFC and BLA did not affect general behavioral changes caused by reduced motivation, but instead prevented alterations in the value of a specific reward from contributing appropriately to cost-benefit decision-making. These results highlight the role of the OFC-BLA pathway in the decision-making process and suggest that communication between these areas is vital for the appropriate assessment of reward value to influence choice.
We used Pavlovian counterconditioning in rats to identify the neural mechanisms for appetitive-aversive motivational interactions. In Stage I, rats were trained on conditioned stimulus (CS)-food (unconditioned stimulus [US]) pairings. In Stage II, this appetitive CS was transformed into a fear CS via pairings with footshock. The development of fear responses was retarded in rats that had received Stage I appetitive training. This counterconditioning was associated with increased levels of phosphorylated mitogen activated protein kinase immunoreactivity (pMAPK-IR) in several brain regions, including midline thalamus, rostral agranular insular cortex (RAIC), lateral amygdala, and nucleus accumbens core and shell, but decreased expression in the ventrolateral quadrant of the midbrain periaqueductal gray. These brain regions showing differential pMAPK-IR have previously been identified as part of the fear prediction error circuit. We then examined the causal role of RAIC MAPK in fear learning and showed that Stage II fear learning was prevented by RAIC infusions of the MEK inhibitor PD098059 (0.5 µg/hemisphere). Taken together, these results show that there are opponent interactions between the appetitive and aversive motivational systems during fear learning and that the transformation of a reward CS into a fear CS is linked to heightened activity in the fear prediction error circuit.
Dopamine is highly implicated both as a teaching signal in reinforcement learning and in motivating actions to obtain rewards. However, theoretical disconnects remain between the temporal encoding properties of dopamine neurons and the behavioral consequences of its release. Here, we demonstrate in rats that dopamine evoked by pavlovian cues increases during acquisition, but dissociates from stable conditioned appetitive behavior as this signal returns to preconditioning levels with extended training. Experimental manipulation of the statistical parameters of the behavioral paradigm revealed that this attenuation of cue-evoked dopamine release during the postasymptotic period was attributable to acquired knowledge of the temporal structure of the task. In parallel, conditioned behavior became less dopamine dependent after extended training. Thus, the current work demonstrates that as the presentation of reward-predictive stimuli becomes anticipated through the acquisition of task information, there is a shift in the neurobiological substrates that mediate the motivational properties of these incentive stimuli.
Evidence indirectly implicates the amygdala as the primary processor of emotional information used by cortex to drive appropriate behavioral responses to stimuli. Taste provides an ideal system with which to test this hypothesis directly, as neurons in both basolateral amygdala (BLA) and gustatory cortex (GC)-anatomically interconnected nodes of the gustatory system-code the emotional valence of taste stimuli (i.e., palatability), in firing rate responses that progress similarly through "epochs." The fact that palatability-related firing appears one epoch earlier in BLA than GC is broadly consistent with the hypothesis that such information may propagate from the former to the latter. Here, we provide evidence supporting this hypothesis, assaying taste responses in small GC single-neuron ensembles before, during, and after temporarily inactivating BLA in awake rats. BLA inactivation (BLAx) changed responses in 98% of taste-responsive GC neurons, altering the entirety of every taste response in many neurons. Most changes involved reductions in firing rate, but regardless of the direction of change, the effect of BLAx was epoch-specific: while firing rates were changed, the taste specificity of responses remained stable; information about taste palatability, however, which normally resides in the "Late" epoch, was reduced in magnitude across the entire GC sample and outright eliminated in most neurons. Only in the specific minority of neurons for which BLAx enhanced responses did palatability specificity survive undiminished. Our data therefore provide direct evidence that BLA is a necessary component of GC gustatory processing, and that cortical palatability processing in particular is, in part, a function of BLA activity.
The role of dopamine in reward is a topic of debate. For example, some have argued that phasic dopamine signaling provides a prediction-error signal necessary for stimulus-reward learning, whereas others have hypothesized that dopamine is not necessary for learning per se, but for attributing incentive motivational value ('incentive salience') to reward cues. These psychological processes are difficult to tease apart, because they tend to change together. To disentangle them we took advantage of natural individual variation in the extent to which reward cues are attributed with incentive salience, and asked whether dopamine (specifically in the core of the nucleus accumbens) is necessary for the expression of two forms of pavlovian-conditioned approach behavior--one in which the cue acquires powerful motivational properties (sign-tracking) and another closely related one in which it does not (goal-tracking). After acquisition of these conditioned responses (CRs), intra-accumbens injection of the dopamine receptor antagonist flupenthixol markedly impaired the expression of a sign-tracking CR, but not a goal-tracking CR. Furthermore, dopamine antagonism did not produce a gradual extinction-like decline in behavior, but maximally impaired expression of a sign-tracking CR on the very first trial, indicating the effect was not due to new learning (i.e. it occurred in the absence of new prediction-error computations). The data support the view that dopamine in the accumbens core is not necessary for learning stimulus-reward associations, but for attributing incentive salience to reward cues, transforming predictive conditional stimuli into incentive stimuli with powerful motivational properties.
If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.
The orbitofrontal cortex (OFC) and amygdala are thought to participate in reversal learning, a process in which cue-outcome associations are switched. However, current theories disagree on whether OFC directs reversal learning in the amygdala. Here, we show that during reversal of cues' associations with rewarding and aversive outcomes, neurons that respond preferentially to stimuli predicting aversive events update more quickly in amygdala than OFC; meanwhile, OFC neurons that respond preferentially to reward-predicting stimuli update more quickly than those in the amygdala. After learning, however, OFC consistently differentiates between impending reinforcements with a shorter latency than the amygdala. Finally, analysis of local field potentials (LFPs) reveals a disproportionate influence of OFC on amygdala that emerges after learning. We propose that reversal learning is supported by complex interactions between neural circuits spanning the amygdala and OFC, rather than directed by any single structure.
Individuals make choices and prioritize goals using complex processes that assign value to rewards and associated stimuli. During Pavlovian learning, previously neutral stimuli that predict rewards can acquire motivational properties, becoming attractive and desirable incentive stimuli. However, whether a cue acts solely as a predictor of reward, or also serves as an incentive stimulus, differs between individuals. Thus, individuals vary in the degree to which cues bias choice and potentially promote maladaptive behaviour. Here we use rats that differ in the incentive motivational properties they attribute to food cues to probe the role of the neurotransmitter dopamine in stimulus-reward learning. We show that intact dopamine transmission is not required for all forms of learning in which reward cues become effective predictors. Rather, dopamine acts selectively in a form of stimulus-reward learning in which incentive salience is assigned to reward cues. In individuals with a propensity for this form of learning, reward cues come to powerfully motivate and control behaviour. This work provides insight into the neurobiology of a form of stimulus-reward learning that confers increased susceptibility to disorders of impulse control.
The attribution of incentive salience to reward-paired cues is dependent on dopamine release in the nucleus accumbens core. These dopamine signals conform to traditional reward-prediction error signals and have been shown to diminish with time. Here we examined if the diminishing dopamine signal in the nucleus accumbens core has functional implications for the expression of sign-tracking, a Pavlovian conditioned response indicative of the attribution of incentive salience to reward-paired cues. Food-restricted male Sprague-Dawley rats were trained in a Pavlovian paradigm in which an insertable lever predicted delivery of food reward in a nearby food cup. After 7 or 14 training sessions, rats received infusions of saline, the dopamine antagonist flupenthixol, or the GABA agonists baclofen and muscimol into the nucleus accumbens core or the dorsal lateral striatum. Dopamine antagonism within the nucleus accumbens core attenuated sign-tracking, whereas reversible inactivation did not affect sign-tracking but increased non-specific food cup checking behaviors. Neither drug in the dorsal lateral striatum affected sign-tracking behavior. Critically, extended training did not alter these effects. Though extended experience with an incentive stimulus may reduce cue-evoked dopamine in the nucleus accumbens core, this does not remove the dependence on dopamine in this region to promote Pavlovian cue approach nor result in the recruitment of dorsal lateral striatal systems for this behavior. These data support the notion that dopamine within the mesoaccumbal system, but not the nigrostriatal system, contributes critically to incentive motivational processes independent of the length of training. This article is protected by copyright. All rights reserved.
Stimuli associated with taking drugs are notorious instigators of relapse. There is, however, considerable variation in the motivational properties of such stimuli, both as a function of the individual and the nature of the stimulus. The behavior of some individuals (sign trackers, STs) is especially influenced by cues paired with reward delivery, perhaps because they are prone to process information via dopamine-dependent, cue-driven, incentive salience systems. Other individuals (goal trackers, GTs) are better able to incorporate higher-order contextual information, perhaps because of better executive/attentional control over behavior, which requires frontal cortical cholinergic activity. We hypothesized, therefore, that a cue that “sets the occasion” for drug taking (a discriminative stimulus, DS) would reinstate cocaine seeking more readily in GTs than STs and that this would require intact cholinergic neurotransmission. To test this, male STs and GTs were trained to self-administer cocaine using an intermittent access schedule with periods of cocaine availability and unavailability signaled by a DS and a DS, respectively. Thereafter, half of the rats received an immunotoxic lesion that destroyed 40-50% of basal forebrain cholinergic neurons and later, after extinction training, were tested for the ability of noncontingent presentations of the DS to reinstate cocaine seeking behavior. The DS was much more effective in reinstating cocaine seeking in GTs than STs and this effect was abolished by cholinergic losses despite the fact that all rats continued to orient to the DS. We conclude that vulnerability to relapse involves interactions between individual cognitive-motivational biases and the form of the drug cue encountered.
Pavlovian conditioned stimuli can acquire incentive motivational properties, and this phenomenon can be measured in animals using Pavlovian conditioned approach behavior. Drugs of abuse can influence the expression of this behavior, and nicotine in particular exhibits incentive amplifying effects. Both conditioned approach behavior and drug abuse rely on overlapping corticolimbic circuitry. We hypothesize that the orbitofrontal cortex (OFC) regulates conditioned approach, and that one site of nicotine action is in the OFC where it reduces cortical output. To test this, we repeatedly exposed rats to 0.4 mg/kg nicotine (s.c.) during training and then pharmacologically inactivated the lateral OFC or performed in vivo electrophysiological recordings of lateral OFC neurons in the presence or absence of nicotine. In Experiment 1, animals were trained in a Pavlovian conditioning paradigm and behavior was evaluated after inactivation of the OFC by microinfusion of the GABA agonists baclofen and muscimol. In Experiment 2, we monitored phasic firing of OFC neurons during Pavlovian conditioning sessions. Nicotine reliably enhanced conditioned responding to the conditioned cue, and inactivation of the OFC reduced conditioned responding, especially the sign-tracking response. OFC neurons exhibited phasic excitations to cue presentation and during goal tracking, and nicotine acutely blunted this phasic neuronal firing. When nicotine was withheld, both conditioned responding and phasic firing in the OFC returned to the level of controls. These results suggest that the OFC is recruited for the expression of conditioned responses, and that nicotine acutely influences this behavior by reducing phasic firing in the OFC.
Autoshaping is a Pavlovian learning paradigm in which rats experience pairings of a CS and a US independently of their behavior. When the CS is a lever inserted into the test cage and the US is food delivered to an adjacent magazine, many rats acquire a lever-pressing response called ‘sign-tracking’ even though that response has no effect on the occurrence of either the CS or the US. Since these lever presses are always followed by the US, it has been suggested that sign-tracking could be due to unintended reinforcement of the response. To eliminate the possibility of such instrumental learning the omission schedule, in which a response to the CS cancels the US, was introduced. Previous research has shown that training rats on autoshaping and switching them to an omission schedule generally reduces but does not eliminate sign-tracking, suggesting that it may be due to both Pavlovian and instrumental learning. In the present study naive rats trained on an omission schedule sign-tracked less than a control group exposed to random, unpaired CS and US presentations, suggesting that they learned to withhold the lever press response because of the negative contingency between that response and the US. In a second experiment rats with dorsal hippocampus lesions sign-tracked more than sham-lesioned rats on omission schedules, suggesting that this case of learning not to respond is hippocampus-based. This conclusion is consistent with many previous findings on the inability of hippocampal rats to withhold or suppress responding, and with studies suggesting that one form of extinction of learned responses in normal rats is due to competition from hippocampus-based learning not to respond.
The attribution of incentive-motivational value to reward-related cues contributes to cue-induced craving and relapse in addicted patients. Recently, it was demonstrated that subanesthetic ketamine increases motivation to quit and decreases cue-induced craving in cocaine-dependent individuals. Although the underlying mechanism of this effect is currently unknown, one possibility is that subanesthetic ketamine decreases the incentive-motivational value of reward-related cues. In the present study, we used a Pavlovian conditioned approach procedure to identify sign-trackers, rats that attribute incentive-motivational value to reward-related cues, and goal-trackers, rats that assign only predictive value to reward-related cues. This model is of interest because sign-trackers are more vulnerable to cue-induced reinstatement of drug-seeking behavior and will persist in this drug-seeking behavior despite adverse consequences. We tested the effect of subanesthetic ketamine on the expression of Pavlovian conditioned approach behavior and the conditioned reinforcing properties of a reward-related cue in sign- and goal-trackers. We found that subanesthetic ketamine decreased sign-tracking and increased goal-tracking behavior in sign-trackers, though it had no effect on conditioned reinforcement. These results suggest that subanesthetic ketamine may be a promising pharmacotherapy for addiction that acts by decreasing the incentive-motivational value of reward-related cues.
Dopamine’s (DA) role in reward-processing is currently discussed as either providing a teaching signal to guide learning or mediating the transfer of incentive salience (i.e. motivational aspects) from unconditioned stimuli (US) to conditioned stimuli (CS). We used a Pavlovian conditioned approach (PCA) procedure to further investigate DAs contribution to these processes. Experiment 1 assessed the acquisition of PCA to a manipulable lever cue for 7 days under DA-blockade with Flupenthixol (FLU; 225 μg/kg) or Saline (SAL) treatment, followed by 6-days off-drug testing. FLU decreased the number of conditioned responses (CR) during the treatment phase, but cessation of treatment resulted in an immediate increase in CR to levels comparable to SAL controls; notably, CR in FLU-treated rats were restricted to goal tracking behaviour. During continued off-drug testing, rats from the FLU group developed sign tracking with a similar temporal pattern as controls. In experiment 2, acquisition of PCA to a non-manipulable auditory cue was investigated. FLU reduced the number of CR during treatment, and removing DA antagonism resulted in a similar rapid increase of CR as seen in experiment 1.
These data complement other reports by demonstrating that, independently from the physical properties of the CS, DA is not required for learning predictive aspects of a CS-US relationship but for the development of behaviour (namely sign tracking) which is based on the motivational aspects of a CS-US relationship.
Individual variation in the attribution of motivational salience to reward-related cues is believed to underlie addiction vulnerability. Pavlovian conditioned approach measures individual variation in motivational salience by identifying rats that are attracted to and motivated by reward cues (sign-trackers) or motivationally fixed on the reward itself (goal-trackers). Previously, it has been demonstrated that sign-trackers are more vulnerable to addiction-like behavior. Moreover, sign-trackers release more dopamine in the nucleus accumbens than goal-trackers in response to reward-related cues, and sign- but not goal-tracking behavior is dopamine-dependent. In the present study, we investigated whether the ventral hippocampus, a potent driver of dopaminergic activity in the nucleus accumbens, modulates the acquisition and expression of Pavlovian conditioned approach behavior. In Experiment 1, lesions of the ventral, but not dorsal or total hippocampus, decreased sign-tracking behavior. In Experiment 2, lesions of the ventral hippocampus did not affect the expression of sign- or goal-tracking behaviors nor conditioned reinforcement. In addition, temporary inactivation of the ventral subiculum, the main output pathway of the ventral hippocampus, did not affect the expression of sign- or goal-tracking behaviors. High-pressure liquid chromatography of nucleus accumbens tissue punches revealed that ventral hippocampal lesions decreased levels of homovanillic acid and the homovanillic acid/dopamine ratio (a marker of dopamine release and metabolism) in only sign-trackers, and decreased accumbal norepinephrine levels in both sign- and goal-trackers. These results suggest that the ventral hippocampus is important for the acquisition but not expression of sign-tracking behavior, possibly as a result of altered dopamine and norepinephrine in the nucleus accumbens. This article is protected by copyright. All rights reserved.
The dorsomedial striatum (DMS) has been implicated in the acquisition of reward representations, a proposal leading to the hypothesis that it should play a role in situations involving reward loss. We report the results of an experiment in which the effects of DMS excitotoxic lesions were tested in consummatory successive negative contrast (reward devaluation), autoshaping training with partial vs. continuous reinforcement (reward uncertainty), and appetitive extinction (reward omission). Animals with DMS lesions exhibited reduced lever pressing responding, but enhanced goal entries, during partial reinforcement training in autoshaping. However, they showed normal negative contrast, acquisition under continuous reinforcement, appetitive extinction, and response facilitation in early extinction trials. Open-field testing also indicated normal motor behavior. Thus, DMS lesions selectively affected the behavioral adjustment to a situation involving reward uncertainty, producing a behavioral reorganization according to which goal tracking (goal entries) became predominant at the expense of sign tracking (lever pressing). This pattern of results shows that the function of the DMS in situations involving reward loss is not general, but restricted to reward uncertainty. We suggest that a nonassociative, drive-related process induced by reward uncertainty requires normal output from DMS neurons.
Learning and motivation are two psychological processes allowing animals to form and express Pavlovian associations between a conditioned stimulus (CS) and an unconditioned stimulus (UCS). However, most models have attempted to capture the mechanisms of learning while neglecting the role that motivation (or incentive salience) may actively play in the expression of behaviour. There is now a body of neurobehavioural evidence showing that incentive salience represents a major determinant of Pavlovian performance. This article presents a motivational model of sign-tracking behaviour whose aim is to explain a wide range of behavioural effects, including those related to partial reinforcement, physiological changes, competition between CSs, and individual differences in responding to a CS. In this model, associative learning is assumed to determine the ability to produce a Pavlovian conditioned response rather than to control the strength and the quality of that response. The model is in keeping with the incentive salience hypothesis and will therefore be discussed in the context of dopamine's role in the brain.
Pavlovian cues for rewards can become attractive incentives: approached and 'wanted' as the rewards themselves. The motivational attractiveness of a previously learned cue is not fixed, but can be dynamically amplified during re-encounter by simultaneous activation of brain limbic circuitry. Here we report that opioid or dopamine microinjections in the dorsolateral quadrant of the neostriatum (DLS) of rats selectively amplify attraction toward a previously learned Pavlovian cue in an individualized fashion, at the expense of a competing cue. In an autoshaping (sign-tracking vs goal-tracking) paradigm, microinjection of the mu opioid receptor agonist (DAMGO) or dopamine indirect agonist (amphetamine) in DLS of sign-tracker individuals selectively enhanced their sign-tracking attraction toward the reward-predictive lever cue. By contrast, DAMGO or amphetamine in DLS of goal-trackers selectively enhanced prepotent attraction toward the reward-proximal cue of sucrose dish. Amphetamine also enhanced goal-tracking in some sign-tracker individuals (if they ever defected to the dish even once). That DLS enhancement of cue attraction was due to stronger motivation, not stronger habits was suggested by: 1) sign-trackers flexibly followed their cue to a new location when the lever was suddenly moved after DLS DAMGO microinjection, and 2) DAMGO in DLS also made sign-trackers work harder on a new instrumental nose-poke response required to earn presentations of their Pavlovian lever cue (instrumental conditioned reinforcement). Altogether, our results suggest that DLS circuitry can enhance the incentive salience of a Pavlovian reward cue, selectively making that cue a stronger motivational magnet. This article is protected by copyright. All rights reserved.
Reciprocal connections between the orbitofrontal cortex and the basolateral nucleus of the amygdala may provide a critical circuit for the learning that underlies goal-directed behavior. We examined neural activity in rat orbitofrontal cortex and basolateral amygdala during instrumental learning in an olfactory discrimination task. Neurons in both regions fired selectively during the anticipation of rewarding or aversive outcomes. This selective activity emerged early in training, before the rats had learned reliably to avoid the aversive outcome. The results support the concept that the basolateral amygdala and orbitofrontal cortex cooperate to encode information that may be used to guide goal-directed behavior.
Cues associated with rewarding events acquire value themselves as a result of the incentive value of the reward being transferred to the cue. Consequently, presentation of a reward-paired cue can trigger reward-seeking behaviors towards the cue itself (i.e., sign-tracking). The ventral pallidum (VP) has been demonstrated to be involved in a number of motivated behaviors, both conditioned and unconditioned. However, its contribution to the acquisition of incentive value is unknown. Using a discriminative autoshaping procedure with levers, we investigated the effects of disrupting VP activity in rats on the emergence of sign-tracking using chemogenetics (Designer Receptors Exclusively Activated by Designer Drugs; DREADDs). Transient disruption of VP neurons (activation of the inhibitory hM4D(Gi) DREADD through systemic injections of clozapine N-oxide (CNO) prior to each autoshaping session) impaired acquisition of sign-tracking (lever press rate) without having any effect on approach to the site of reward delivery (i.e., goal-tracking) nor on the expression of sign-tracking after it was acquired. In addition, we conducted electrophysiological recordings in freely behaving rats following VP DREADD activation. The majority of VP units that were responsive to CNO injections exhibited rapid inhibition relative to baseline, a subset of CNO-responsive units showed delayed excitation, and a smaller subset displayed a mixed response of inhibition and excitation following CNO injections. We argue that disruption of VP during autoshaping specifically disrupted the transfer of incentive value that was attributed to the lever cue, suggesting a surprisingly fundamental role for the VP in acquiring compared to expressing Pavlovian incentive values. This article is protected by copyright. All rights reserved.
Recently evidence has emerged suggesting a role for the paraventricular nucleus of the thalamus (PVT) in the processing of reward-associated cues. However, the specific role of the PVT in these processes has yet to be elucidated. Here we use an animal model that captures individual variation in response to discrete reward-associated cues to further assess the role of the PVT in stimulus-reward learning. When rats are exposed to a Pavlovian conditioning paradigm, wherein a discrete cue predicts food reward, two distinct conditioned responses emerge. Some rats, termed sign-trackers, approach and manipulate the cue; whereas others, termed goal-trackers, approach the location of reward delivery upon cue presentation. For both sign- and goal-trackers the cue is a predictor; but only for sign-trackers is it also an incentive stimulus. We investigated the role of the PVT in the acquisition and expression of these conditioned responses using an excitotoxic lesion. Results indicate that PVT lesions prior to acquisition amplify the differences between phenotypes-increasing sign-tracking and attenuating goal-tracking behavior. Lesions of the PVT after rats had acquired their respective conditioned responses also attenuated the expression of the goal-tracking response, and increased the sign-tracking response, but did so selectively in goal-trackers. These results suggest that the PVT acts to suppress the attribution of incentive salience to reward cues, as disruption of the functional activity within this structure enhances the tendency to sign-track. This article is protected by copyright. All rights reserved.
This article is protected by copyright. All rights reserved.
A cue associated with a rewarding event can trigger behavior towards the cue itself due to the cue acquiring incentive value through its pairing with the rewarding outcome (i.e., sign-tracking). For example, rats will approach, press, and attempt to "consume" a retractable lever conditioned stimulus (CS) that signals delivery of a food unconditioned stimulus (US). Attending to food-predictive CSs is important when seeking out food, and it is just as important to be able to modify one's behavior when the relationships between CSs and USs are changed. Using a discriminative autoshaping procedure with lever CSs, the present study investigated the effects of orbitofrontal cortex (OFC) lesions on sign-tracking and reversal learning. Insertion of one lever was followed by sucrose delivery upon retraction, and insertion of another lever was followed by nothing. After the acquisition phase, the contingencies between the levers and outcomes were reversed. Bilateral OFC lesions had no effect on the acquisition of sign-tracking. However, OFC-lesioned rats showed substantial deficits in acquiring sign-tracking compared to sham-lesioned rats once the stimulus-outcome contingencies were reversed. Over the course of reversal learning, OFC-lesioned rats were able to reach comparable levels of sign-tracking as sham-lesioned rats. These findings suggest that OFC is not necessary for the ability of a CS to acquire incentive value and provide more evidence that OFC is critical for modifying behavior appropriately following a change in stimulus-outcome contingencies.
Dopaminergic signals play a mathematically precise role in reward-related learning, and variations in dopaminergic signaling have been implicated in vulnerability to addiction. Here, we provide a detailed overview of the relationship between theoretical, mathematical, and experimental accounts of phasic dopamine signaling, with implications for the role of learning-related dopamine signaling in addiction and related disorders. We describe the theoretical and behavioral characteristics of model-free learning based on errors in the prediction of reward, including step-by-step explanations of the underlying equations. We then use recent insights from an animal model that highlights individual variation in learning during a Pavlovian conditioning paradigm to describe overlapping aspects of incentive salience attribution and model-free learning. We argue that this provides a computationally coherent account of some features of addiction.
We examined the contribution of the amygdala to value signals within orbital prefrontal cortex (OFC) and medial prefrontal cortex (MFC). On each trial, monkeys chose between two stimuli that were associated with different quantities of reward. In intact monkeys, as expected, neurons in both OFC and MFC signaled the reward quantity associated with stimuli. Contrasted with MFC, OFC contained a larger proportion of neurons encoding reward quantity and did so with faster response latencies. Removing the amygdala eliminated these differences, mainly by decreasing value coding in OFC. Similar decreases occurred in OFC immediately before and after reward delivery. Although the amygdala projects to both OFC and MFC, we found that it has its greatest influence over reward-value coding in OFC. Notably, amygdala lesions did not abolish value coding in OFC, which shows that OFC's representations of the value of objects, choices, and outcomes depends, in large part, on other sources.
Certain Pavlovian conditioned stimuli (CSs) paired with food unconditioned stimuli (USs) come to elicit approach and even consumption-like behaviors in rats (sign-tracking). We investigated the effects of lesions of the nucleus accumbens core (ACbC) or shell (ACbS) on the acquisition of sign-tracking in a discriminative autoshaping procedure in which presentation of one lever CS was followed by delivery of sucrose, and another was not. Although we previously found that bilateral lesions of the whole ACb disrupted the initial acquisition of sign-tracking, neither ACbC or ACbS lesions affected the rate or percentage of trials in which rats pressed the CS+. In addition, detailed video analysis showed no effect of either lesion on the topography of the sign-tracking conditioned response (CR). These and other results from lesion studies of autoshaping contrast with those from previous sign-tracking experiments that used purely visual cues (Parkinson et al., 2000a,b), suggesting that the neural circuitry involved in assigning incentive value depends upon the nature of the CS.
Animals are not passive spectators of the sensory world in which they live. In natural conditions they often sense objects on the bases of expectations initiated by predictive cues. Expectation profoundly modulates neural activity by altering the background state of cortical networks and modulating sensory processing. The link between these two effects is not known. Here, we studied how cue-triggered expectation of stimulus availability influences processing of sensory stimuli in the gustatory cortex (GC). We found that expected tastants were coded more rapidly than unexpected stimuli. The faster onset of sensory coding related to anticipatory priming of GC by associative auditory cues. Simultaneous recordings and pharmacological manipulations of GC and basolateral amygdala revealed the role of top-down inputs in mediating the effects of anticipatory cues. Altogether, these data provide a model for how cue-triggered expectation changes the state of sensory cortices to achieve rapid processing of natural stimuli.
Initially-neutral cues paired with rewards are thought to acquire motivational significance, as if the incentive motivational value of the reward is transferred to the cue. Such cues may serve as secondary reinforcers to establish new learning, modulate the performance of instrumental action (Pavlovian-instrumental transfer, PIT), and be the targets of approach and other cue-directed behaviors. Here we examined the effects of lesions of the ventral striatal nucleus accumbens (ACb) and the basolateral amygdala (BLA) on the acquisition of discriminative autoshaped lever-pressing in rats. Insertion of one lever into the experimental chamber was reinforced by sucrose delivery, but insertion of another lever was not reinforced. Although sucrose was delivered independently of the rats' behavior, sham-lesioned rats rapidly came to press the reinforced but not the nonreinforced lever. Bilateral ACb lesions impaired the initial acquisition of sign-tracking but not its terminal levels. In contrast, BLA lesions produced substantial deficits in terminal levels of sign-tracking. Furthermore, whereas ACb lesions primarily affected the probability of lever press responses, BLA lesions mostly affected the rate of responding once it occurred. Finally, disconnection lesions that disrupted communication between ACb and BLA produced both sets of deficits. We suggest that ACb is important for initial acquisition of consummatory-like responses that incorporate hedonic aspects of the reward, while BLA serves to enhance such incentive salience once it is acquired.
Neutral cues paired with rewards often appear to acquire motivational significance, as if the incentive motivational value of the reward is transferred to the cue. Such cues have been reported to modulate the performance of instrumental action (Pavlovian-instrumental transfer, PIT), serve as conditioned reinforcers in the establishment of new learning, and be the targets of approach and other cue-directed behaviors. Here we examined the effects of lesions of the amygdala central nucleus (CeA) on the acquisition of discriminative autoshaped lever-pressing. Insertion of one lever into the experimental chamber was reinforced by sucrose delivery, but insertion of another lever was not reinforced. Although sucrose delivery was not contingent on lever pressing, both CeA- and sham-lesioned rats rapidly came to press the reinforced but not the nonreinforced lever. Despite their showing little evidence of impairments in autoshaped lever pressing, these same CeA-lesioned rats showed significant deficits in the expression of PIT in a subsequent phase of the experiment. The lack of impaired autoshaping in CeA-lesioned rats contrasts with effects previously reported for conditioned orienting responses (ORs) and for other putative measures of incentive learning including PIT and conditioned approach to visual cues.