Computational roles for dopamine in behavioral control

Harvard University, Cambridge, Massachusetts, United States
Nature (Impact Factor: 41.46). 11/2004; 431(7010):760-7. DOI: 10.1038/nature03015
Source: PubMed


Neuromodulators such as dopamine have a central role in cognitive disorders. In the past decade, biological findings on dopamine function have been infused with concepts taken from computational theories of reinforcement learning. These more abstract approaches have now been applied to describe the biological algorithms at play in our brains when we form value judgements and make choices. The application of such quantitative models has opened up new fields, ripe for attack by young synthesizers and theoreticians.

  • Source
    • "Finally, while we observed significant c-Fos levels in the VTA we did not detect any difference between control and sucrose-exposed rats. The dopamine system plays a central role in reward processes and is one of the main afferent regions projecting to the NAc (Berridge & Robinson, 1998;Schultz, 2000;Montague et al., 2004). Several studies using microdialysis or fast-scan voltammetry report an increase in dopamine release in the NAc during consumption or intraoral injection of sweet solutions (Hajnal & Norgren, 2001;Bassareo et al., 2002;Hajnal et al., 2004;Roitman et al., 2008). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Adolescence is a critical period characterized by major neurobiological changes. Chronic stimulation of the reward system might constitute an important factor in vulnerability to pathological development. In spite of the dramatic increase in the consumption of sweet palatable foods during adolescence in our modern societies, the long-term consequences of such exposure on brain reward processing remain poorly understood. Here, we investigated in rats the long-lasting effects of sugar overconsumption during their adolescence on their adult reactivity to the hedonic properties of sweet rewards. Adolescent rats with continuous access to 5% sucrose solution (from postnatal day 30-46) showed escalating intake. At adulthood (post-natal day 70), using two-bottle free choice tests, sucrose-exposed rats showed lower intake than non-exposed rats suggesting decreased sensitivity to the rewarding properties of sucrose. In Experiment 1, we tested their hedonic-related orofacial reactions to intraoral infusion of tasty solutions. We showed that sucrose-exposed rats presented less hedonic reactions in response to sweet tastes leaving the reactivity to water or quinine unaltered. Hence, in Experiment 2, we observed that this hedonic deficit is associated with lower c-Fos expression levels in the nucleus accumbens, a brain region known to play a central role in hedonic processing. These findings demonstrate that a history of high sucrose intake during the critical period of adolescence induces long-lasting deficits in hedonic treatment that may contribute to reward-related disorders.
    Full-text · Article · Jan 2016 · European Journal of Neuroscience
    • "Thus, a biological/ behavioral marker from an inexpensive, easily administered behavioral test would be highly useful if it could add in the prediction of conversion , especially if it were sensitive to perturbation of the dopamine system . Measures of reinforcement learning (RL) are a logical candidate to consider in this context, based on the large basic neuroscience literature pointing to a critical role for dopamine in RL and motivation (see Montague et al., 2004; Schultz, 2013, for reviews). More specifically, in typically functioning people, dopamine cell firing increases after the receipt of a better-than-expected outcome, whereas the same cells transiently stop firing after worse-than-expected outcome. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Early identification efforts for psychosis have thus far yielded many more individuals "at risk" than actually develop psychotic illness. Here, we test whether measures of reinforcement learning (RL), known to be impaired in chronic schizophrenia, are related to the severity of clinical risk symptoms. Because of the reliance of RL on dopamine-rich frontostriatal systems and evidence of dopamine system dysfunction in the psychosis prodrome, RL measures are of specific interest in this clinical population. The current study examines relationships between psychosis risk symptoms and RL task performance in a sample of adolescents and young adults (n = 70) receiving mental health services. We observed significant correlations between multiple measures of RL performance and measures of both positive and negative symptoms. These results suggest that RL measures may provide a psychosis risk signal in treatment-seeking youth. Further research is necessary to understand the potential predictive role of RL measures for conversion to psychosis.
    No preview · Article · Nov 2015 · Journal of Nervous & Mental Disease
    • "This approach is used in many operant learning paradigms, such as probabilistic response (or Go/NoGo) learning (Frank and O'Reilly 2006; Holroyd et al. 2004; Pessiglione et al. 2006), probabilistic stimulus selection (PSS; Frank et al. 2004; Shanks et al. 2002), and probabilistic reversal learning (Cools et al. 2002). Learning, in the context of such paradigms, is driven by mismatches between expected and obtained outcomes, called reward prediction errors (RPEs; Glimcher 2011; Montague et al. 2004). Choices that lead to better-than-expected outcomes, or positive RPEs, facilitate those choices in those contexts—a process called " Go-learning " (Frank et al. 2004). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Motivational deficits (avolition and anhedonia ) have historically been considered important negative symptoms of schizophrenia (SZ). Numerous studies have attempted to identify the neural substrates of avolition and anhedonia in schizophrenia , but these studies have not produced much agreement. Deficits in various aspects of reinforcement processing have been observed in individuals with schizophrenia, but it is not exactly clear which of these deficits actually engender motivational impairments in SZ. The purpose of this chapter is to examine how various reinforcement-related behavioral and neural signals could contribute to motivational impairments in both schizophrenia and psychiatric illness, in general. In particular, we describe different aspects of the concept of expected value (EV) , such as the distinction between the EV of stimuli and the expected value of actions, the acquisition of value versus the estimation of value, and the discounting of value as a consequence of time or effort required. We conclude that avolition and anhedonia in SZ are most commonly tied to aberrant signals for expected value, in the context of learning. We discuss implications for further research on the neural substrates of motivational impairments in psychiatric illness.
    No preview · Article · Sep 2015 · Current Topics in Behavioral Neurosciences
Show more