Dissociable responses to punishment in distinct striatal regions during reversal learning

Department of Psychiatry and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, Addenbrooke's Hospital, P. O. Box 189, Level E4, Hills Road, Cambridge, CB2 2QQ, UK.
NeuroImage (Impact Factor: 6.36). 03/2010; 51(4):1459-67. DOI: 10.1016/j.neuroimage.2010.03.036
Source: PubMed


Adaptive behavior depends on the ability to flexibly alter our choices in response to changes in reward and punishment contingencies. One brain region frequently implicated in such behavior is the striatum. However, this region is functionally diverse and there are a number of apparent inconsistencies across previous studies. For instance, how can significant BOLD responses in the ventral striatum during punishment-based reversal learning be reconciled with the frequently demonstrated role of the ventral striatum in reward processing? Here we attempt to address this question by separately examining BOLD responses during reversal learning driven by reward and during reversal learning driven by punishment. We demonstrate simultaneous valence-specific and valence-nonspecific signals in the striatum, with the posterior dorsal striatum responding only to unexpected reward, and the anterior ventral striatum responding to both unexpected punishment as well as unexpected reward. These data help to reconcile conflicting findings from previous studies by showing that distinct regions of the striatum exhibit dissociable responses to punishment during reversal learning.

Download full-text


Available from: Oliver J Robinson,
    • "Results further demonstrate that aversive prediction error-related activity was enhanced in the hippocampus, insula and ventral striatum (Fig. 2/B1). The involvement of both striatum and insula is consistent with previous studies, where they have repeatedly been implicated as the neural correlates of aversive prediction errors (Delgado et al., 2008a; Metereau and Dreher, 2013; Robinson et al., 2010; Schiller et al., 2008; Seymour et al., 2004, 2007; Spoormaker et al., 2011). The strongest enhancement of aversive prediction error-related activity, however, was found in the right hippocampus (Fig. 2/B1; Table 2). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Cognitive emotion regulation is a powerful way of modulating emotional responses. However, despite the vital role of emotions in learning, it is unknown whether the effect of cognitive emotion regulation also extends to the modulation of learning. Computational models indicate prediction error activity, typically observed in the striatum and ventral tegmental area, as a critical neural mechanism involved in associative learning. We used model-based fMRI during aversive conditioning with and without cognitive emotion regulation to test the hypothesis that emotion regulation would affect prediction error-related neural activity in the striatum and ventral tegmental area, reflecting an emotion regulation-related modulation of learning. Our results show that cognitive emotion regulation reduced emotion-related brain activity, but increased prediction error-related activity in a network involving ventral tegmental area, hippocampus, insula and ventral striatum. While the reduction of response activity was related to behavioral measures of emotion regulation success, the enhancement of prediction error-related neural activity was related to learning performance. Furthermore, functional connectivity between the ventral tegmental area and ventrolateral prefrontal cortex, an area involved in regulation, was specifically increased during emotion regulation and likewise related to learning performance. Our data, therefore, provide first-time evidence that beyond reducing emotional responses, cognitive emotion regulation affects learning by enhancing prediction error-related activity, potentially via tegmental dopaminergic pathways. Copyright © 2015. Published by Elsevier Inc.
    NeuroImage 08/2015; 123. DOI:10.1016/j.neuroimage.2015.08.038 · 6.36 Impact Factor
  • Source
    • "Dopamine is implicated in shifting response set, including triggering of a behavioral switch (Cools, Lewis et al. 2006; Lee et al. 2007), learning new associations (Shohamy et al. 2005), maintenance of learned associations (Cohen et al. 2002), commission of action (Frank et al. 2004), and both tracking and responding to rewarding or punishing (i.e., hedonic or valenced) outcomes in decision making (Cools et al. 2009; van der Schaaf et al. 2014). Several brain regions, all of which receive strong dopaminergic innervation , are thought to support this process, including medial and lateral prefrontal cortex as well as the striatum (Fellows and Farah 2003; Izquierdo et al. 2004; Robinson et al. 2010; Rygula et al. 2010; Hampshire et al. 2012; Fallon et al. 2013). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Dopamine is implicated in multiple functions, including motor execution, action learning for hedonically salient outcomes, maintenance, and switching of behavioral response set. Here, we used a novel within-subject psychopharmacological and combined functional neuroimaging paradigm, investigating the interaction between hedonic salience, dopamine, and response set shifting, distinct from effects on action learning or motor execution. We asked whether behavioral performance in response set shifting depends on the hedonic salience of reversal cues, by presenting these as null (neutral) or salient (monetary loss) outcomes. We observed marked effects of reversal cue salience on set-switching, with more efficient reversals following salient loss outcomes. l-Dopa degraded this discrimination, leading to inappropriate perseveration. Generic activation in thalamus, insula, and striatum preceded response set switches, with an opposite pattern in ventromedial prefrontal cortex (vmPFC). However, the behavioral effect of hedonic salience was reflected in differential vmPFC deactivation following salient relative to null reversal cues. l-Dopa reversed this pattern in vmPFC, suggesting that its behavioral effects are due to disruption of the stability and switching of firing patterns in prefrontal cortex. Our findings provide a potential neurobiological explanation for paradoxical phenomena, including maintenance of behavioral set despite negative outcomes, seen in impulse control disorders in Parkinson's disease.
    Cerebral Cortex 09/2014; DOI:10.1093/cercor/bhu210 · 8.67 Impact Factor
  • Source
    • "Previous studies on reinforcement learning have found distinct neural mechanisms for learning by reward and that by punishment [36], [37], [39], [73]. In particular, although several studies on reward and punishment reinforcement learning also used a reversal learning paradigm [37], [73], these studies focused on prediction error and the dopamine system in the striatum, which is very different from the way we analyzed the data. Similarly, although lesion studies have suggested that the ventromedial PFC was responsible for reversal learning [17], [74], especially that driven by negative feedback [35], these studies did not directly compare reversal learning by reward and punishment. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Impairments in flexible goal-directed decisions, often examined by reversal learning, are associated with behavioral abnormalities characterized by impulsiveness and disinhibition. Although the lateral orbital frontal cortex (OFC) has been consistently implicated in reversal learning, it is still unclear whether this region is involved in negative feedback processing, behavioral control, or both, and whether reward and punishment might have different effects on lateral OFC involvement. Using a relatively large sample (N = 47), and a categorical learning task with either monetary reward or moderate electric shock as feedback, we found overlapping activations in the right lateral OFC (and adjacent insula) for reward and punishment reversal learning when comparing correct reversal trials with correct acquisition trials, whereas we found overlapping activations in the right dorsolateral prefrontal cortex (DLPFC) when negative feedback signaled contingency change. The right lateral OFC and DLPFC also showed greater sensitivity to punishment than did their left homologues, indicating an asymmetry in how punishment is processed. We propose that the right lateral OFC and anterior insula are important for transforming affective feedback to behavioral adjustment, whereas the right DLPFC is involved in higher level attention control. These results provide insight into the neural mechanisms of reversal learning and behavioral flexibility, which can be leveraged to understand risky behaviors among vulnerable populations.
    PLoS ONE 12/2013; 8(12):e82169. DOI:10.1371/journal.pone.0082169 · 3.23 Impact Factor
Show more