Overlapping Prediction Errors in Dorsal Striatum During Instrumental Learning With Juice and Money Reward in the Human Brain

Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, California, USA.
Journal of Neurophysiology (Impact Factor: 3.04). 09/2009; 102(6):3384-91. DOI: 10.1152/jn.91195.2008
Source: PubMed

ABSTRACT Prediction error signals have been reported in human imaging studies in target areas of dopamine neurons such as ventral and dorsal striatum during learning with many different types of reinforcers. However, a key question that has yet to be addressed is whether prediction error signals recruit distinct or overlapping regions of striatum and elsewhere during learning with different types of reward. To address this, we scanned 17 healthy subjects with functional magnetic resonance imaging while they chose actions to obtain either a pleasant juice reward (1 ml apple juice), or a monetary gain (5 cents) and applied a computational reinforcement learning model to subjects' behavioral and imaging data. Evidence for an overlapping prediction error signal during learning with juice and money rewards was found in a region of dorsal striatum (caudate nucleus), while prediction error signals in a subregion of ventral striatum were significantly stronger during learning with money but not juice reward. These results provide evidence for partially overlapping reward prediction signals for different types of appetitive reinforcers within the striatum, a finding with important implications for understanding the nature of associative encoding in the striatum as a function of reinforcer type.

Download full-text


Available from: Vivian Virag Valentin, Aug 21, 2014
  • Source
    • "Money, a secondary reinforcer, was chosen for use in the study given that it can be somewhat equated in the positive and negative domains (i.e., it can be gained or lost, unlike many primary reinforcers). Neuroimaging studies in humans have highlighted a role for the striatum in Pavlovian conditioning with appetitive and aversive primary reinforcers (e.g., juice: O'Doherty et al., 2004; shock: see Phelps and LeDoux, 2005, for review), and also with secondary reinforcers (e.g., money: Kirsch et al., 2003; Valentin and O'Doherty, 2009; Delgado et al., 2011). Thus, we hypothesized that the striatum would be engaged in our simple Pavlovian conditioning paradigm with secondary reinforcers. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Pavlovian conditioning involves the association of an inherently neutral stimulus with an appetitive or aversive outcome, such that the neutral stimulus itself acquires reinforcing properties. Across species, this type of learning has been shown to involve subcortical brain regions such as the striatum and the amygdala. It is less clear, however, how the neural circuitry involved in the acquisition of Pavlovian contingencies in humans, particularly in the striatum, is affected by acute stress. In the current study, we investigate the effect of acute stress exposure on Pavlovian conditioning using monetary reinforcers. Participants underwent a partial reinforcement conditioning procedure in which neutral stimuli were paired with high and low magnitude monetary gains and losses. A between-subjects design was used, such that half of the participants were exposed to cold stress while the remaining participants were exposed to a no stress control procedure. Cortisol measurements and subjective ratings were used as measures of stress. We observed an interaction between stress, valence, and magnitude in the ventral striatum, with the peak in the putamen. More specifically, the stress group exhibited an increased sensitivity to magnitude in the gain domain. This effect was driven by those participants who experienced a larger increase in circulating cortisol levels in response to the stress manipulation. Taken together, these results suggest that acute stress can lead to individual differences in circulating cortisol levels which influence the striatum during Pavlovian conditioning with monetary reinforcers.
    Frontiers in Behavioral Neuroscience 05/2014; 8:179. DOI:10.3389/fnbeh.2014.00179 · 4.16 Impact Factor
  • Source
    • "The effectiveness of this type of negative feedback might be complicated by the ethical consideration that subjects should never lose money [35], [39]. Still, although overlapping neural mechanisms for reward and reward prediction errors have been identified for primary and secondary rewards using money and juice [44], [45], it is not clear whether monetary loss is analogous to a primary punishment, such as air puff or electric shock in driving behavioral flexibility. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Impairments in flexible goal-directed decisions, often examined by reversal learning, are associated with behavioral abnormalities characterized by impulsiveness and disinhibition. Although the lateral orbital frontal cortex (OFC) has been consistently implicated in reversal learning, it is still unclear whether this region is involved in negative feedback processing, behavioral control, or both, and whether reward and punishment might have different effects on lateral OFC involvement. Using a relatively large sample (N = 47), and a categorical learning task with either monetary reward or moderate electric shock as feedback, we found overlapping activations in the right lateral OFC (and adjacent insula) for reward and punishment reversal learning when comparing correct reversal trials with correct acquisition trials, whereas we found overlapping activations in the right dorsolateral prefrontal cortex (DLPFC) when negative feedback signaled contingency change. The right lateral OFC and DLPFC also showed greater sensitivity to punishment than did their left homologues, indicating an asymmetry in how punishment is processed. We propose that the right lateral OFC and anterior insula are important for transforming affective feedback to behavioral adjustment, whereas the right DLPFC is involved in higher level attention control. These results provide insight into the neural mechanisms of reversal learning and behavioral flexibility, which can be leveraged to understand risky behaviors among vulnerable populations.
    PLoS ONE 12/2013; 8(12):e82169. DOI:10.1371/journal.pone.0082169 · 3.23 Impact Factor
  • Source
    • "One possible explanation for VTA/SN and NA activations in our study is the difference between primary and secondary rewards. For example, one fMRI study reported that NA activations were associated only with the processing of primary rewards (Beck et al. 2010), whereas another fMRI study showed significant NA activations during the processing of secondary rewards (Valentin and O'Doherty 2009). There is also fMRI evidence linking NA activations to coding the values of both primary and secondary rewards (Sescousse et al. 2010). "
    [Show abstract] [Hide abstract]
    ABSTRACT: The motivation of getting rewards or avoiding punishments reinforces learning behaviors. Although the neural mechanisms underlying the effect of rewards on episodic memory have been demonstrated, there is little evidence of the effect of punishments on this memory. Our functional magnetic resonance imaging (fMRI) study investigated the effects of monetary rewards and punishments on activation during the encoding of source memories. During encoding, participants memorized words (item) and locations of presented words (source) under 3 conditions (Reward, Punishment, and Control). During retrieval, participants retrieved item and source memories of the words and were rewarded or penalized according to their performance. Source memories encoded with rewards or punishments were remembered better than those without such encoding. fMRI data demonstrated that the ventral tegmental area and substantia nigra and nucleus accumbens activations reflected both the processes of reward and punishment, whereas insular activation increased as a linear function of punishment. Activation in the hippocampus and parahippocampal cortex predicted subsequent retrieval success of source memories. Additionally, correlations between these reward/punishment-related regions and the hippocampus were significant. The successful encoding of source memories could be enhanced by punishments and rewards, and interactions between reward/punishment-related regions and memory-related regions could contribute to memory enhancement by reward and/or punishment.
    Cerebral Cortex 01/2013; 24(5). DOI:10.1093/cercor/bhs415 · 8.67 Impact Factor
Show more