Separate Valuation Subsystems for Delay and Effort Decision Costs

Cognitive Neuroscience Center, Centre National de la Recherche Scientifique, Unité mixte de recherche 5229, Reward and Decision Making Team, 69675 Bron, France.
The Journal of Neuroscience : The Official Journal of the Society for Neuroscience (Impact Factor: 6.34). 10/2010; 30(42):14080-90. DOI: 10.1523/JNEUROSCI.2752-10.2010
Source: PubMed

ABSTRACT Decision making consists of choosing among available options on the basis of a valuation of their potential costs and benefits. Most theoretical models of decision making in behavioral economics, psychology, and computer science propose that the desirability of outcomes expected from alternative options can be quantified by utility functions. These utility functions allow a decision maker to assign subjective values to each option under consideration by weighting the likely benefits and costs resulting from an action and to select the one with the highest subjective value. Here, we used model-based neuroimaging to test whether the human brain uses separate valuation systems for rewards (erotic stimuli) associated with different types of costs, namely, delay and effort. We show that humans devalue rewards associated with physical effort in a strikingly similar fashion to those they devalue that are associated with delays, and that a single computational model derived from economics theory can account for the behavior observed in both delay discounting and effort discounting. However, our neuroimaging data reveal that the human brain uses distinct valuation subsystems for different types of costs, reflecting in opposite fashion delayed reward and future energetic expenses. The ventral striatum and the ventromedial prefrontal cortex represent the increasing subjective value of delayed rewards, whereas a distinct network, composed of the anterior cingulate cortex and the anterior insula, represent the decreasing value of the effortful option, coding the expected expense of energy. Together, these data demonstrate that the valuation processes underlying different types of costs can be fractionated at the cerebral level.

Download full-text


Available from: Jean-Claude Dreher, Sep 29, 2015
32 Reads
  • Source
    • "negative variation; CRN, cue-related negativity; DRL, Driven Right Leg; EEG, electroencephalography ; ERN, error-related negativity; ERP, event-related potential; FRN, feedback-related negativity; RM-ANOVA, repeated measures analysis of variance; SCP, statistical cluster plot; UDTR, up-down transformed-rule. Neuroscience 273 (2014) 100–117 and anterior cingulate cortex have been implicated in task monitoring when participants made errors while bidding for rewards (Hare et al., 2008), when participants evaluate task effort needed to obtain primary rewards (Prevost et al., 2010), and when evaluating conflict between high-risk or low-risk choices (Kuhnen and Knutson, 2005). Neuroimaging, however, is not ideal for examining the interaction of processes that may occur over very fast timescales. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Task execution almost always occurs in the context of reward-seeking or punishment-avoiding behavior. As such, ongoing task monitoring systems are influenced by reward anticipation systems. In turn, when a task has been executed either successfully or unsuccessfully, future iterations of that task will be re-titrated on the basis of the task outcome. Here, we examined the neural underpinnings of the task-monitoring and reward-evaluation systems to better understand how they govern reward seeking behavior. Twenty-three healthy adult participants performed a task where they accrued points that equated to real world value (gift cards) by responding as rapidly as possible within an allotted timeframe, while success rate was titrated online by changing the duration of the timeframe dependent on participant performance. Informative cues initiated each trial, indicating the probability of potential reward or loss (four levels from very low to very high). We manipulated feedback by first informing participants of task success/failure, after which a second feedback signal indicated actual magnitude of reward/loss. High-density EEG recordings allowed for examination of event-related potentials (ERPs) to the informative cues and in turn, to both feedback signals. Distinct ERP components associated with reward cues, task preparatory and task monitoring processes, and reward feedback processes were identified. Unsurprisingly, participants displayed increased ERP amplitudes associated with task preparatory processes following cues that predicted higher chances of reward. They also rapidly updated reward and loss prediction information dependent on task performance after the first feedback signal. Finally, upon reward receipt, initial reward probability was no longer taken into account. Rather, ERP measures suggested that only the magnitude of actual reward or loss was now processed. Reward and task monitoring processes are clearly dissociable, but interact across very fast timescales to update reward predictions as information about task success or failure is accrued. Careful delineation of these processes will be useful in future investigations in clinical groups where such processes are suspected of having gone awry.
    Neuroscience 05/2014; 273. DOI:10.1016/j.neuroscience.2014.05.002 · 3.36 Impact Factor
  • Source
    • "The insular cortex may play a role in shifting between these behavioral strategies. For, this structure has been implicated in interoceptive awareness, homeostatic control and energy expenditure (Butti and Hof, 2010; Prévost et al., 2010; Craig, 2011). Furthermore, the insular cortex has connections with the dorsal and ventral striatum and thereby may exert an effect on immediate and long-term focus (see Tanaka et al., 2004). "
    [Show abstract] [Hide abstract]
    ABSTRACT: In the Iowa Gambling Task (IGT) subjects need to find a way to earn money in a context of variable wins and losses, conflicting short-term and long-term pay-off, and uncertainty of outcomes. In 2006, we published the first rodent version of the IGT (r-IGT; Behavior Research Methods 38, 470-478). Here, we discuss emerging ideas on the involvement of different prefrontal-striatal networks in task-progression in the r-IGT, as revealed by our studies thus far. The emotional system, encompassing, among others, the orbitofrontal cortex, infralimbic cortex and nucleus accumbens (shell and core area), may be involved in assessing and anticipating the value of different options in the early stages of the task, i.e., as animals explore and learn task contingencies. The cognitive control system, encompassing, among others, the prelimbic cortex and dorsomedial striatum, may be involved in instrumental goal-directed behavior in later stages of the task, i.e., as behavior toward long-term options is strengthened (reinforced) and behavior toward long-term poor options is weakened (punished). In addition, we suggest two directions for future research: (1) the role of the internal state of the subject in decision-making, and (2) studying differences in task-related costs. Overall, our studies have contributed to understanding the interaction between the emotional system and cognitive control system as crucial to navigating human and non-human animals alike through a world of variable wins and losses, conflicting short-term and long-term pay-offs, and uncertainty of outcomes.
    Frontiers in Psychology 03/2014; 5:203. DOI:10.3389/fpsyg.2014.00203 · 2.80 Impact Factor
  • Source
    • "In conclusion, our results provide support for the idea that different regions of frontal cortex mediate different forms of cost-benefit decision making, as has previously been suggested (Rudebeck et al., 2006; Walton et al., 2006; Floresco et al., 2008; Prevost et al., 2010). The ACC clearly plays a role in effort-reward decisions involving ramp climbing (e.g., Walton et al., 2003) and possibly pressing levers multiple times [but see Schweimer and Hauber (2005), Walton et al. (2009)]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The rat anterior cingulate cortex (ACC) mediates effort-based decision making when the task requires the physical effort of climbing a ramp. Normal rats will readily climb a barrier leading to high reward whereas rats with ACC lesions will opt instead for an easily obtained small reward. The present study explored whether the role of ACC in cost-benefit decisions extends beyond climbing by testing its role in ramp climbing as well as two novel cost-benefit decision tasks, one involving the physical effort of lifting weights and the other the emotional cost of overcoming fear (i.e., "courage"). As expected, rats with extensive ACC lesions tested on a ramp-climbing task were less likely to choose a high-reward/high-effort arm than sham controls. However, during the first few trials, lesioned rats were as likely as controls to initially turn into the high-reward arm (HRA) but far less likely to actually climb the barrier, suggesting that the role of the ACC is not in deciding which course of action to pursue, but rather in maintaining a course of action in the face of countervailing forces. In the effort-reward decision task involving weight lifting, some lesion animals behaved like controls while others avoided the HRA. However, the results were not statistically significant and a follow-up study using incremental increasing effort failed to show any difference between lesion and control groups. The results suggest that the ACC is not needed for effort-reward decisions involving weight lifting but may affect motor abilities. Finally, a courage task explored the willingness of rats to overcome the fear of crossing an open, exposed arm to obtain a high reward. Both sham and ACC-lesioned animals exhibited equal tendencies to enter the open arm. However, whereas sham animals gradually improved on the task, ACC-lesioned rats did not. Taken together, the results suggest that the role of the ACC in effort-reward decisions may be limited to certain tasks.
    Frontiers in Behavioral Neuroscience 01/2014; 8:12. DOI:10.3389/fnbeh.2014.00012 · 3.27 Impact Factor
Show more