Overlapping Prediction Errors in Dorsal Striatum During Instrumental Learning With Juice and Money Reward in the Human Brain

Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, California, USA.
Journal of Neurophysiology (Impact Factor: 2.89). 09/2009; 102(6):3384-91. DOI: 10.1152/jn.91195.2008
Source: PubMed


Prediction error signals have been reported in human imaging studies in target areas of dopamine neurons such as ventral and dorsal striatum during learning with many different types of reinforcers. However, a key question that has yet to be addressed is whether prediction error signals recruit distinct or overlapping regions of striatum and elsewhere during learning with different types of reward. To address this, we scanned 17 healthy subjects with functional magnetic resonance imaging while they chose actions to obtain either a pleasant juice reward (1 ml apple juice), or a monetary gain (5 cents) and applied a computational reinforcement learning model to subjects' behavioral and imaging data. Evidence for an overlapping prediction error signal during learning with juice and money rewards was found in a region of dorsal striatum (caudate nucleus), while prediction error signals in a subregion of ventral striatum were significantly stronger during learning with money but not juice reward. These results provide evidence for partially overlapping reward prediction signals for different types of appetitive reinforcers within the striatum, a finding with important implications for understanding the nature of associative encoding in the striatum as a function of reinforcer type.

Download full-text


Available from: Vivian Virag Valentin, Aug 21, 2014
18 Reads
  • Source
    • "Uncertain premiums have some economic appeal to marketers, since they need not be paid out for each smaller-sized portion choice. However, prior research on the commensurability of food with nonfood resources has focused only on premiums whose receipt is certain to occur (e.g., Kim et al. 2011; Valentin and O'Doherty 2009). Observing that the Pilot Study results are replicated when the receipt of the premium is uncertain would thus add to this literature. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Four studies show that offering consumers the choice between a full-sized food portion alone and a half-sized food portion paired with a small non-food premium (e.g., a small Happy Meal toy or the mere possibility of winning frequent flyer miles) motivates smaller portion choice. We also find that both food and the prospect of receiving a non-food premium activate a common area of the brain (the striatum), which is associated with reward, desire, and motivation. Finally, we show that the choice results are mediated by a psychological desire for (motivation to obtain), but not by liking of, the monetary premium. Notably, we find that choice of the smaller food portion is most pronounced when the probability of obtaining the premium is not disclosed compared to when the probability is disclosed or when the receipt of the same premium is stated as being certain.
  • Source
    • "Money, a secondary reinforcer, was chosen for use in the study given that it can be somewhat equated in the positive and negative domains (i.e., it can be gained or lost, unlike many primary reinforcers). Neuroimaging studies in humans have highlighted a role for the striatum in Pavlovian conditioning with appetitive and aversive primary reinforcers (e.g., juice: O'Doherty et al., 2004; shock: see Phelps and LeDoux, 2005, for review), and also with secondary reinforcers (e.g., money: Kirsch et al., 2003; Valentin and O'Doherty, 2009; Delgado et al., 2011). Thus, we hypothesized that the striatum would be engaged in our simple Pavlovian conditioning paradigm with secondary reinforcers. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Pavlovian conditioning involves the association of an inherently neutral stimulus with an appetitive or aversive outcome, such that the neutral stimulus itself acquires reinforcing properties. Across species, this type of learning has been shown to involve subcortical brain regions such as the striatum and the amygdala. It is less clear, however, how the neural circuitry involved in the acquisition of Pavlovian contingencies in humans, particularly in the striatum, is affected by acute stress. In the current study, we investigate the effect of acute stress exposure on Pavlovian conditioning using monetary reinforcers. Participants underwent a partial reinforcement conditioning procedure in which neutral stimuli were paired with high and low magnitude monetary gains and losses. A between-subjects design was used, such that half of the participants were exposed to cold stress while the remaining participants were exposed to a no stress control procedure. Cortisol measurements and subjective ratings were used as measures of stress. We observed an interaction between stress, valence, and magnitude in the ventral striatum, with the peak in the putamen. More specifically, the stress group exhibited an increased sensitivity to magnitude in the gain domain. This effect was driven by those participants who experienced a larger increase in circulating cortisol levels in response to the stress manipulation. Taken together, these results suggest that acute stress can lead to individual differences in circulating cortisol levels which influence the striatum during Pavlovian conditioning with monetary reinforcers.
    Frontiers in Behavioral Neuroscience 05/2014; 8:179. DOI:10.3389/fnbeh.2014.00179 · 3.27 Impact Factor
  • Source
    • "The effectiveness of this type of negative feedback might be complicated by the ethical consideration that subjects should never lose money [35], [39]. Still, although overlapping neural mechanisms for reward and reward prediction errors have been identified for primary and secondary rewards using money and juice [44], [45], it is not clear whether monetary loss is analogous to a primary punishment, such as air puff or electric shock in driving behavioral flexibility. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Impairments in flexible goal-directed decisions, often examined by reversal learning, are associated with behavioral abnormalities characterized by impulsiveness and disinhibition. Although the lateral orbital frontal cortex (OFC) has been consistently implicated in reversal learning, it is still unclear whether this region is involved in negative feedback processing, behavioral control, or both, and whether reward and punishment might have different effects on lateral OFC involvement. Using a relatively large sample (N = 47), and a categorical learning task with either monetary reward or moderate electric shock as feedback, we found overlapping activations in the right lateral OFC (and adjacent insula) for reward and punishment reversal learning when comparing correct reversal trials with correct acquisition trials, whereas we found overlapping activations in the right dorsolateral prefrontal cortex (DLPFC) when negative feedback signaled contingency change. The right lateral OFC and DLPFC also showed greater sensitivity to punishment than did their left homologues, indicating an asymmetry in how punishment is processed. We propose that the right lateral OFC and anterior insula are important for transforming affective feedback to behavioral adjustment, whereas the right DLPFC is involved in higher level attention control. These results provide insight into the neural mechanisms of reversal learning and behavioral flexibility, which can be leveraged to understand risky behaviors among vulnerable populations.
    PLoS ONE 12/2013; 8(12):e82169. DOI:10.1371/journal.pone.0082169 · 3.23 Impact Factor
Show more