Available via license: CC BY 4.0
Content may be subject to copyright.
A preview of the PDF is not available
Dopamine neurons drive fear extinction learning by signaling the omission of expected aversive outcomes
Abstract
Extinction of fear responses is critical for adaptive behavior and deficits in this form of safety learning are hallmark of anxiety disorders. However, the neuronal mechanisms that initiate extinction learning are largely unknown. Here we show, using single-unit electrophysiology and cell-type specific fiber photometry, that dopamine neurons in the ventral tegmental area (VTA) are activated by the omission of the aversive unconditioned stimulus (US) during fear extinction. This dopamine signal occurred specifically during the beginning of extinction when the US omission is unexpected, and correlated strongly with extinction learning. Furthermore, temporally-specific optogenetic inhibition or excitation of dopamine neurons at the time of the US omission revealed that this dopamine signal is both necessary for, and sufficient to accelerate, normal fear extinction learning. These results identify a prediction error-like neuronal signal that is necessary to initiate fear extinction and reveal a crucial role of DA neurons in this form of safety learning.
... Several recent studies, including one from our group, report that dopaminergic neurons are activated in operant tasks and regulate taskrelated decision-making 4,7,41 . Emerging evidence also suggests that dopaminergic activity is triggered by and possibly regulates cueassociative aversive behavior 5,33,42 . We therefore examined dopaminergic activity in the auditory striatum and its possible role in auditory fear conditioning. ...
The auditory striatum, a sensory portion of the dorsal striatum, plays an essential role in learning and memory. In contrast to its roles and underlying mechanisms in operant conditioning, however, little is known about its contribution to classical auditory fear conditioning. Here, we reveal the function of the auditory striatum in auditory-conditioned fear memory. We find that optogenetically inhibiting auditory striatal neurons impairs fear memory formation, which is mediated through the striatal-amygdala pathway. Using calcium imaging in behaving mice, we find that auditory striatal neuronal responses to conditioned tones potentiate across memory acquisition and expression. Furthermore, nigrostriatal dopaminergic projections plays an important role in modulating conditioning-induced striatal potentiation. Together, these findings demonstrate the existence of a nigro-striatal-amygdala circuit for conditioned fear memory formation and expression.
... Furthermore, it is noteworthy that the mPFC extinction engram neurons also receive synaptic inputs from the mediodorsal nucleus of the thalamus (MD) and the ventral tegmental area (VTA). The MD is considered to play a regulatory role in fear extinction [89], while the VTA is implicated in the generation of bottom-up prediction error signals [90,91]. These inputs suggest that the formation of extinction memory engrams may involve the integration of multiple bottom-up signals by the mPFC engram neurons. ...
Fear extinction is a biological process in which learned fear behavior diminishes without anticipated reinforcement, allowing the organism to re-adapt to ever-changing situations. Based on the behavioral hypothesis that extinction is new learning and forms an extinction memory, this new memory is more readily forgettable than the original fear memory. The brain’s cellular and synaptic traces underpinning this inherently fragile yet reinforceable extinction memory remain unclear. Intriguing questions are about the whereabouts of the engram neurons that emerged during extinction learning and how they constitute a dynamically evolving functional construct that works in concert to store and express the extinction memory. In this review, we discuss recent advances in the engram circuits and their neural connectivity plasticity for fear extinction, aiming to establish a conceptual framework for understanding the dynamic competition between fear and extinction memories in adaptive control of conditioned fear responses.
Cerebellar networks have traditionally been linked to sensorimotor control. However, a large body of evidence suggests that cerebellar functions extend to non-motor realms, such as fear-based emotional processing and that these functions are supported by interactions with a wide range of brain structures. Research related to the cerebellar contributions to emotional processing has focussed primarily on the use of well-constrained conditioning paradigms in both human and non-human subjects. From these studies, cerebellar circuits appear to be critically involved in both conditioned and unconditioned responses to threatening stimuli in addition to encoding and storage of fear memory. It has been hypothesised that the computational mechanism underlying this contribution may involve internal models, where errors between actual and expected outcomes are computed within the circuitry of the cerebellum. From a clinical perspective, cerebellar abnormalities have been consistently linked to neurodevelopmental disorders, including autism. Importantly, atypical adaptive behaviour and heightened anxiety are also common amongst autistic individuals. In this review, we provide an overview of the current anatomical, physiological and theoretical understanding of cerebellar contributions to fear-based emotional processing to foster further insights into the neural circuitry underlying emotional dysregulation observed in people with autism.
Chronic pain often develops severe mood changes such as depression. However, how chronic pain leads to depression remains elusive and the mechanisms determining individuals’ responses to depression are largely unexplored. Here we found that depression-like behaviors could only be observed in 67.9% of mice with chronic neuropathic pain, leaving 32.1% of mice with depression resilience. We determined that the spike discharges of the ventral tegmental area (VTA)-projecting lateral habenula (LHb) glutamatergic (Glu) neurons were sequentially increased in sham, resilient and susceptible mice, which consequently inhibited VTA dopaminergic (DA) neurons through a LHbGlu-VTAGABA-VTADA circuit. Furthermore, the LHbGlu-VTADA excitatory inputs were dampened via GABAB receptors in a pre-synaptic manner. Regulation of LHb-VTA pathway largely affected the development of depressive symptoms caused by chronic pain. Our study thus identifies a pivotal role of the LHb-VTA pathway in coupling chronic pain with depression and highlights the activity-dependent contribution of LHbGlu-to-VTADA inhibition in depressive behavioral regulation.
Dopamine, a catecholamine neurotransmitter, has historically been associated with the encoding of reward, whereas its role in aversion has received less attention. Here, we systematically gathered the vast evidence of the role of dopamine in the simplest forms of aversive learning: classical fear conditioning and extinction. In the past, crude methods were used to augment or inhibit dopamine to study its relationship with fear conditioning and extinction. More advanced techniques such as conditional genetic, chemogenic and optogenetic approaches now provide causal evidence for dopamine's role in these learning processes. Dopamine neurons encode conditioned stimuli during fear conditioning and extinction and convey the signal via activation of D 1–4 receptor sites particularly in the amygdala, prefrontal cortex and striatum. The coordinated activation of dopamine receptors allows for the continuous formation, consolidation, retrieval and updating of fear and extinction memory in a dynamic and reciprocal manner. Based on the reviewed literature, we conclude that dopamine is crucial for the encoding of classical fear conditioning and extinction and contributes in a way that is comparable to its role in encoding reward.
Relief, the appetitive state after the termination of aversive stimuli, is evolutionarily conserved. Understanding the behavioral role of this well-conserved phenomenon and its underlying neurobiological mechanisms are open and important questions. Here, we discover that the magnitude of relief from physical stress strongly correlates with individual resilience to depression-like behaviors in chronic stressed mice. Notably, blocking stress relief causes vulnerability to depression-like behaviors, whereas natural rewards supplied shortly after stress promotes resilience. Stress relief is mediated by reward-related mesolimbic dopamine neurons, which show minute-long, persistent activation after stress termination. Circuitry-wise, activation or inhibition of circuits downstream of the ventral tegmental area during the transient relief period bi-directionally regulates depression resilience. These results reveal an evolutionary function of stress relief in depression resilience and identify the neural substrate mediating this effect. Importantly, our data suggest a behavioral strategy of augmenting positive valence of stress relief with natural rewards to prevent depression.
Simple Summary
Threat conditioning is a process by which animals learn about danger. As such, this process has been extensively studied as a model of how humans develop fears and anxiety, and their related disorders such as post-traumatic stress disorder (PTSD), which affects seven to eight percent of adults at some point in their life. Conversely, threat extinction is essentially the reverse of this process, by which animals adaptively learn that something is no longer dangerous. Threat extinction forms the basis of exposure therapy, a cornerstone in the treatment of such disorders, during which patients face their fears in order to overcome them. So-called higher-order forms of learning such as second-order threat conditioning add layers of complexity to threat conditioning and may thus more closely mimic processes underlying the development of such disorders. While the amygdala has long been thought to be at the center of such processes, being activated and suppressed during threat conditioning and extinction, respectively, in reality a complex network of structures governs these forms of learning. While such networks are not proposed to hold all the answers to our understanding of the development and treatment of such disorders, they are an important piece of the puzzle, and are thus reviewed here. Recent advances are presenting new areas of exploration into the brain-based mechanisms of such forms of learning and may ultimately inform the treatment of fear- and anxiety-related disorders.
Abstract
Threat conditioning, extinction, and second-order threat conditioning studied in animal models provide insight into the brain-based mechanisms of fear- and anxiety-related disorders and their treatment. Much attention has been paid to the role of the basolateral amygdala (BLA) in such processes, an overview of which is presented in this review. More recent evidence suggests that the BLA serves as the core of a greater network of structures in these forms of learning, including associative and sensory cortices. The BLA is importantly regulated by hippocampal and prefrontal inputs, as well as by the catecholaminergic neuromodulators, norepinephrine and dopamine, that may provide important prediction-error or learning signals for these forms of learning. The sensory cortices may be required for the long-term storage of threat memories. As such, future research may further investigate the potential of the sensory cortices for the long-term storage of extinction and second-order conditioning memories.
Excessive avoidance and slow extinction of fear are hallmarks of anxiety disorders. We have previously found that overnight fasting diminishes excessive avoidance and speeds up fear extinction by decreasing subjective relief during threat omissions. Since relief tracks the reward prediction error signal that governs safety learning, we hypothesized that these effects of fasting might be linked to a decreased activation in brain regions related to reward prediction error processing. Hence, we replicated our previous study in a 3T-MRI scanner. Overnight fasting increased effective avoidance and sped up fear extinction learning. During extinction, the fasting group showed lower activations in the ventromedial prefrontal cortex and nucleus accumbens in response to threat omissions signaled by a safe cue. Nucleus accumbens activations were modulated by relief in the control group. This study provides support for overnight fasting as an adjunct to treatments for anxiety, but the effects should be investigated in anxious patients.
Accurately predicting an outcome requires that animals learn supporting and conflicting evidence from sequential experience. In mammals and invertebrates, learned fear responses can be suppressed by experiencing predictive cues without punishment, a process called memory extinction. Here, we show that extinction of aversive memories in Drosophila requires specific dopaminergic neurons, which indicate that omission of punishment is remembered as a positive experience. Functional imaging revealed co-existence of intracellular calcium traces in different places in the mushroom body output neuron network for both the original aversive memory and a new appetitive extinction memory. Light and ultrastructural anatomy are consistent with parallel competing memories being combined within mushroom body output neurons that direct avoidance. Indeed, extinction-evoked plasticity in a pair of these neurons neutralizes the potentiated odor response imposed in the network by aversive learning. Therefore, flies track the accuracy of learned expectations by accumulating and integrating memories of conflicting events.
Midbrain dopamine neurons are well known for their role in reward-based reinforcement learning. We found that the activity of dopamine axons in the posterior tail of the striatum (TS) scaled with the novelty and intensity of external stimuli, but did not encode reward value. We demonstrated that the ablation of TS-projecting dopamine neurons specifically inhibited avoidance of novel or high-intensity stimuli without affecting animals' initial avoidance responses, suggesting a role in reinforcement rather than simply in avoidance itself. Furthermore, we found that animals avoided optogenetic activation of dopamine axons in TS during a choice task and that this stimulation could partially reinstate avoidance of a familiar object. These results suggest that TS-projecting dopamine neurons reinforce avoidance of threatening stimuli. More generally, our results indicate that there are at least two axes of reinforcement learning using dopamine in the striatum: one based on value and one based on external threat.
Dopamine neurons are thought to encode novelty in addition to reward prediction error (the discrepancy between actual and predicted values). In this study, we compared dopamine activity across the striatum using fiber fluorometry in mice. During classical conditioning, we observed opposite dynamics in dopamine axon signals in the ventral striatum (‘VS dopamine’) and the posterior tail of the striatum (‘TS dopamine’). TS dopamine showed strong excitation to novel cues, whereas VS dopamine showed no responses to novel cues until they had been paired with a reward. TS dopamine cue responses decreased over time, depending on what the cue predicted. Additionally, TS dopamine showed excitation to several types of stimuli including rewarding, aversive, and neutral stimuli whereas VS dopamine showed excitation only to reward or reward-predicting cues. Together, these results demonstrate that dopamine novelty signals are localized in TS along with general salience signals, while VS dopamine reliably encodes reward prediction error.
Functional neuroanatomy of Pavlovian fear has identified neuronal circuits and synapses associating conditioned stimuli with aversive events. Hebbian plasticity within these networks requires additional reinforcement to store particularly salient experiences into long-term memory. Here we have identified a circuit that reciprocally connects the ventral periaqueductal gray and dorsal raphe region with the central amygdala and that gates fear learning. We found that ventral periaqueductal gray and dorsal raphe dopaminergic (vPdRD) neurons encode a positive prediction error in response to unpredicted shocks and may reshape intra-amygdala connectivity via a dopamine-dependent form of long-term potentiation. Negative feedback from the central amygdala to vPdRD neurons might limit reinforcement to events that have not been predicted. These findings add a new module to the midbrain dopaminergic circuit architecture underlying associative reinforcement learning and identify vPdRD neurons as a critical component of Pavlovian fear conditioning. We propose that dysregulation of vPdRD neuronal activity may contribute to fear-related psychiatric disorders.
Overcoming aversive emotional memories requires neural systems that detect when fear responses are no longer appropriate so that they can be extinguished. The midbrain ventral tegmental area (VTA) dopamine system has been implicated in reward and more broadly in signaling when a better-than-expected outcome has occurred. This suggests that it may be important in guiding fear to safety transitions. We report that when an expected aversive outcome does not occur, activity in midbrain dopamine neurons is necessary to extinguish behavioral fear responses and engage molecular signaling events in extinction learning circuits. Furthermore, a specific dopamine projection to the nucleus accumbens medial shell is partially responsible for this effect. In contrast, a separate dopamine projection to the medial prefrontal cortex opposes extinction learning. This demonstrates a novel function for the canonical VTA-dopamine reward system and reveals opposing behavioral roles for different dopamine neuron projections in fear extinction learning.
ELife digest
There are many types of learning; one type of learning means that rewards and punishments can shape future behavior. Dopamine is a molecule that allows neurons in the brain to communicate with one another, and it is released in response to unexpected rewards. Most neuroscientists believe that dopamine is important to learn from the reward; however, there are different opinions about whether dopamine is important to learn from punishments or not.
Previous studies that tried to examine how dopamine activities change in response to punishment have reported different results. One of the likely reasons for the controversy is that it was difficult to measure only the activity of dopamine-releasing neurons.
To overcome this issue, Matsumoto et al. used genetically engineered mice in which shining a blue light into their brain would activate their dopamine neurons but not any other neurons. Tiny electrodes were inserted into the brains of these mice, and a blue light was used to confirm that these electrodes were recording from the dopamine-producing neurons. Specifically if the electrode detected an electrical impulse when blue light was beamed into the brain, then the recorded neuron was confirmed to be a dopamine-producing neuron.
Measuring the activities of these dopamine neurons revealed that they were indeed activated by reward but inhibited by punishment. In other words, dopamine neurons indeed can signal punishments as negative and rewards as positive on a single axis. Further experiments showed that, if the mice predicted both a reward and a punishment, the dopamine neurons could integrate information from both to direct learning.
Matsumoto et al. also saw that when mice received rewards too often, their dopamine neurons did not signal punishment correctly. These results suggest that how we feel about punishment may depend on how often we experience rewards.
In addition to learning, dopamine has also been linked to many psychiatric symptoms such as addiction and depression. The next challenge will be to examine how the frequency of rewards changes an animal’s state and responses to punishment in more detail, and how this relates to normal and abnormal behaviors.
DOI: http://dx.doi.org/10.7554/eLife.17328.002
The amygdala, a critical structure for both Pavlovian fear conditioning and fear extinction, receives sparse but comprehensive dopamine innervation and contains dopamine D1 and D2 receptors. Fear extinction, which involves learning to suppress the expression of a previously learned fear, appears to require the dopaminergic system. The specific roles of D2 receptors in mediating associative learning underlying fear extinction require further study. Intra-basolateral amygdala (BLA) infusions of a D2 receptor agonist, quinpirole, and a D2 receptor antagonist, sulpiride, prior to fear extinction and extinction retention were tested 24 h after fear extinction training for long-term memory (LTM). LTM was facilitated by quinpirole and attenuated by sulpiride. In addition, A-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor glutamate receptor 1 (GluR1) subunit, GluR1 phospho-Ser845, and N-methyl-D-aspartic acid receptor NR2B subunit levels in the BLA were generally increased by quinpirole and down-regulated by sulpiride. The present study suggests that activation of D2 receptors facilitates fear extinction and that blockade of D2 receptors impairs fear extinction, accompanied by changes in GluR1, GluR1-Ser845 and NR2B levels in the amygdala.
Generalized fear is a maladaptive behavior in which non-threatening stimuli elicit a fearful response. Here, we demonstrate that discrimination between predictive and non-predictive threat stimuli is highly sensitive to probabilistic discounting and increasing threat intensity in mice. We find that dopamine neurons of the ventral tegmental area (VTA) encode both the negative valence of threat-predictive cues and the certainty of threat prediction. As fear generalization emerges, the dopamine neurons that are activated by a threat predictive cue (CS+) decrease the amplitude of activation and an equivalent signal emerges to a non-predictive cue (CS-). Temporally precise enhancement of dopamine neurons during threat conditioning to high threat levels or uncertain threats can prevent generalization. Moreover, phasic enhancement of genetically captured dopamine neurons activated by threat cues can reverse fear generalization. These findings demonstrate the dopamine neurons reflect the certainty of threat prediction that can be used to inform and update the fear engram.
The relief from an aversive event is rewarding. Since organisms are able to learn which environmental cues can cease an aversive event, relief learning helps to better cope with future aversive events. Literature data suggest that relief learning is affected in various psychopathological conditions, such as anxiety disorders. Here, we investigated the role of the mesolimbic dopamine system in relief learning. Using a relief learning procedure in Sprague Dawley rats, we applied a combination of behavioral experiments with anatomical tracing, c-Fos immunohistochemistry, and local chemogenetic and pharmacological interventions to broadly characterize the role of the mesolimbic dopamine system. The present study shows that a specific part of the mesolimbic dopamine system, the projection from the posterior medial ventral tegmental area (pmVTA) to the nucleus accumbens shell (AcbSh), is activated by aversive electric stimuli. 6-OHDA lesions of the pmVTA blocked relief learning but fear learning and safety learning were not affected. Chemogenetic silencing of the pmVTA-AcbSh projection using the DREADD approach, as well as intra-AcbSh injections of the dopamine D2/3 receptor antagonist raclopride inhibited relief learning. Taken together, the present data demonstrate that the dopaminergic pmVTA-AcbSh projection is critical for relief learning but not for similar learning phenomena. This novel finding may have clinical implications since the processing of signals predicting relief and safety is often impaired in patients suffering from anxiety disorders. Furthermore, it may help to better understand psychological conditions like non-suicidal self-injury, which are associated with pain offset relief.
Anxiety disorders constitute the largest group of mental disorders in most western societies and are a leading cause of disability. The essential features of anxiety disorders are excessive and enduring fear, anxiety or avoidance of perceived threats, and can also include panic attacks. Although the neurobiology of individual anxiety disorders is largely unknown, some generalizations have been identified for most disorders, such as alterations in the limbic system, dysfunction of the hypothalamic-pituitary-adrenal axis and genetic factors. In addition, general risk factors for anxiety disorders include female sex and a family history of anxiety, although disorder-specific risk factors have also been identified. The diagnostic criteria for anxiety disorders varies for the individual disorders, but are generally similar across the two most common classification systems: The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) and the International Classification of Diseases, Tenth Edition (ICD-10). Despite their public health significance, the vast majority of anxiety disorders remain undetected and untreated by health care systems, even in economically advanced countries. If untreated, these disorders are usually chronic with waxing and waning symptoms. Impairments associated with anxiety disorders range from limitations in role functioning to severe disabilities, such as the patient being unable to leave their home.