Explicit neural signals reflecting reward uncertainty

Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge CB2 3DY, UK.
Philosophical Transactions of The Royal Society B Biological Sciences (Impact Factor: 6.31). 11/2008; 363(1511):3801-11. DOI: 10.1098/rstb.2008.0152
Source: PubMed

ABSTRACT The acknowledged importance of uncertainty in economic decision making has stimulated the search for neural signals that could influence learning and inform decision mechanisms. Current views distinguish two forms of uncertainty, namely risk and ambiguity, depending on whether the probability distributions of outcomes are known or unknown. Behavioural neurophysiological studies on dopamine neurons revealed a risk signal, which covaried with the standard deviation or variance of the magnitude of juice rewards and occurred separately from reward value coding. Human imaging studies identified similarly distinct risk signals for monetary rewards in the striatum and orbitofrontal cortex (OFC), thus fulfilling a requirement for the mean variance approach of economic decision theory. The orbitofrontal risk signal covaried with individual risk attitudes, possibly explaining individual differences in risk perception and risky decision making. Ambiguous gambles with incomplete probabilistic information induced stronger brain signals than risky gambles in OFC and amygdala, suggesting that the brain's reward system signals the partial lack of information. The brain can use the uncertainty signals to assess the uncertainty of rewards, influence learning, modulate the value of uncertain rewards and make appropriate behavioural choices between only partly known options.

Download full-text


Available from: Colin Farrell Camerer, Jul 28, 2015
  • Source
    • "We have studied reward learning in a passive learning situation. It is known that existing RPE schemes do not fully account for learning in this setting (Dayan and Niv, 2008; Schultz and Dickinson, 2000): For example, they have limited capacity for subjective uncertainty (Preuschoff and Bossaerts, 2007; Schultz et al., 2008) and simply associate each cue or " state " with a single value. Experimental evidence points to simple learning in the absence of RPEs, e.g. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Surprise drives learning. Various neural “prediction error” signals are believed to underpin surprise-based reinforcement learning. Here, we report a surprise signal that reflects reinforcement learning but is neither un/signed reward prediction error (RPE) nor un/signed state prediction error (SPE). To exclude these alternatives, we measured surprise responses in the absence of RPE and accounted for a host of potential SPE confounds. This new surprise signal was evident in ventral striatum, primary sensory cortex, frontal poles, and amygdala. We interpret these findings via a normative model of surprise. Hum Brain Mapp, 2014. © 2014 Wiley Periodicals, Inc.
    Human Brain Mapping 09/2014; 35(9). DOI:10.1002/hbm.22513 · 6.92 Impact Factor
  • Source
    • "These regions show a rapid phasic response to cues of incentive in animal studies with single cell recording and also in human functional magnetic resonance imaging (fMRI) research. These regions are also engaged in processing information regarding the receipt (or omission) of rewards (Rangel et al., 2008; Schulz et al., 2008). Research on the Nacc suggests that a key function of the reward system is motivating effort towards reward. "
    [Show abstract] [Hide abstract]
    ABSTRACT: A series of studies suggest that bipolar disorder is related to high sensitivity to incentives and that incentive sensitivity (or sensitivity of the approach system) can predict the course of mania. Incentive sensitivity in bipolar disorder seems to be related to two processes: a tendency to invest in difficult-to-attain goals and an over-reactivity to cues of goal progress versus thwarting. Both of those processes appear relevant to symptom generation. Hence, bipolar disorder seems related to a greater emphasis on reaching goals and also a problematic reactivity to reaching those highly desired goals. We suggest directions for treatment development focused on these issues in goal regulation.
    Clinical Psychology & Psychotherapy 07/2012; 19(4):352-62. DOI:10.1002/cpp.1801 · 2.59 Impact Factor
  • Source
    • "food units) is equivalent to constant supply (here, 4 food units every time); there is no unpredictability and therefore no risk [3]. But, as mentioned above, and contrary to the current opinion [3] [21] [23], I defend that unpredictable variance induces uncertainty rather than risk. Thirdly, and importantly for my purpose, uncertainty only becomes a source of risk when a decision maker's own resources are limited. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Determining how living beings react to tasks that reflect realistic situations of risk has given rise to a vast literature. However, I argue that the methodologies traditionally used to test humans and nonhumans relative to risk often fail to achieve their goal. When risk is modelled in laboratory, potential decision cost (or potential loss) typically denotes an absence of optimal gain. In contrast, when risk occurs in real-life situations, potential loss denotes the reduction in an individual's limited resources - whether energetic, social, financial, etc. This conceptual difference about the nature of risk may have important implications for the understanding of the parameters that control risk-taking behaviour.
    Behavioural brain research 04/2012; 229(2):443-6. DOI:10.1016/j.bbr.2012.01.032 · 3.39 Impact Factor
Show more