Wolfram Schultz

University of Cambridge, Cambridge, England, United Kingdom

Are you Wolfram Schultz?

Claim your profile

Publications (152)985.18 Total impact

  • Martin D Vestergaard, Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Accurate retrospection is critical in many decision scenarios ranging from investment banking to hedonic psychology. A notoriously difficult case is to integrate previously perceived values over the duration of an experience. Failure in retrospective evaluation leads to suboptimal outcome when previous experiences are under consideration for revisit. A biologically plausible mechanism underlying evaluation of temporally extended outcomes is leaky integration of evidence. The leaky integrator favours positive temporal contrasts, in turn leading to undue emphasis on recency. To investigate choice mechanisms underlying suboptimal outcome based on retrospective evaluation, we used computational and behavioural techniques to model choice between perceived extended outcomes with different temporal profiles. Second-price auctions served to establish the perceived values of virtual coins offered sequentially to humans in a rapid monetary gambling task. Results show that lesser-valued options involving successive growth were systematically preferred to better options with declining temporal profiles. The disadvantageous inclination towards persistent growth was mitigated in some individuals in whom a longer time constant of the leaky integrator resulted in fewer violations of dominance. These results demonstrate how focusing on immediate gains is less beneficial than considering longer perspectives.
    Proceedings of the Royal Society B: Biological Sciences 07/2015; 282(1810). DOI:10.1098/rspb.2014.1766 · 5.29 Impact Factor
  • Source
    The Journal of Neuroscience : The Official Journal of the Society for Neuroscience 02/2015; 35(7):3146-3154. · 6.75 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Economic choices are largely determined by two principal elements, reward value (utility) and probability. Although nonlinear utility functions have been acknowledged for centuries, nonlinear probability weighting (probability distortion) was only recently recognized as a ubiquitous aspect of real-world choice behavior. Even when outcome probabilities are known and acknowledged, human decision makers often overweight low probability outcomes and underweight high probability outcomes. Whereas recent studies measured utility functions and their corresponding neural correlates in monkeys, it is not known whether monkeys distort probability in a manner similar to humans. Therefore, we investigated economic choices in macaque monkeys for evidence of probability distortion. We trained two monkeys to predict reward from probabilistic gambles with constant outcome values (0.5 ml or nothing). The probability of winning was conveyed using explicit visual cues (sector stimuli). Choices between the gambles revealed that the monkeys used the explicit probability information to make meaningful decisions. Using these cues, we measured probability distortion from choices between the gambles and safe rewards. Parametric modeling of the choices revealed classic probability weighting functions with inverted-S shape. Therefore, the animals overweighted low probability rewards and underweighted high probability rewards. Empirical investigation of the behavior verified that the choices were best explained by a combination of nonlinear value and nonlinear probability distortion. Together, these results suggest that probability distortion may reflect evolutionarily preserved neuronal processing. Copyright © 2015 Stauffer et al.
    The Journal of Neuroscience : The Official Journal of the Society for Neuroscience 02/2015; 37(7):3146-3154. DOI:10.1523/JNEUROSCI.3653-14.2015 · 6.75 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The best rewards are often distant and can only be achieved by planning and decision-making over several steps. We designed a multi-step choice task in which monkeys followed internal plans to save rewards toward self-defined goals. During this self-controlled behavior, amygdala neurons showed future-oriented activity that reflected the animal's plan to obtain specific rewards several trials ahead. This prospective activity encoded crucial components of the animal's plan, including value and length of the planned choice sequence. It began on initial trials when a plan would be formed, reappeared step by step until reward receipt, and readily updated with a new sequence. It predicted performance, including errors, and typically disappeared during instructed behavior. Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior. The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding
    Nature Neuroscience 01/2015; advance online publication. · 14.98 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The best rewards are often distant and can only be achieved by planning and decision-making over several steps. We designed a multi-step choice task in which monkeys followed internal plans to save rewards toward self-defined goals. During this self-controlled behavior, amygdala neurons showed future-oriented activity that reflected the animal's plan to obtain specific rewards several trials ahead. This prospective activity encoded crucial components of the animal's plan, including value and length of the planned choice sequence. It began on initial trials when a plan would be formed, reappeared step by step until reward receipt, and readily updated with a new sequence. It predicted performance, including errors, and typically disappeared during instructed behavior. Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior. The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.
    Nature Neuroscience 01/2015; DOI:10.1038/nn.3925 · 14.98 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Although there is a rich literature on the role of dopamine in value learning, much less is known about its role in using established value estimations to shape decision-making. Here we investigated the effect of dopaminergic modulation on value-based decision-making for food items in fasted healthy human participants. The Becker-deGroot-Marschak auction, which assesses subjective value, was examined in conjunction with pharmacological fMRI using a dopaminergic agonist and an antagonist. We found that dopamine enhanced the neural response to value in the inferior parietal gyrus/intraparietal sulcus, and that this effect predominated toward the end of the valuation process when an action was needed to record the value. Our results suggest that dopamine is involved in acting upon the decision, providing additional insight to the mechanisms underlying impaired decision-making in healthy individuals and clinical populations with reduced dopamine levels. Copyright © 2014 the authors 0270-6474/14/3416856-09$15.00/0.
    The Journal of Neuroscience : The Official Journal of the Society for Neuroscience 12/2014; 34(50):16856-64. DOI:10.1523/JNEUROSCI.2051-14.2014 · 6.75 Impact Factor
  • Source
    William R Stauffer, Armin Lak, Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Optimal choices require an accurate neuronal representation of economic value. In economics, utility functions are mathematical representations of subjective value that can be constructed from choices under risk. Utility usually exhibits a nonlinear relationship to physical reward value that corresponds to risk attitudes and reflects the increasing or decreasing marginal utility obtained with each additional unit of reward. Accordingly, neuronal reward responses coding utility should robustly reflect this nonlinearity. Results In two monkeys, we measured utility as a function of physical reward value from meaningful choices under risk (that adhered to first- and second-order stochastic dominance). The resulting nonlinear utility functions predicted the certainty equivalents for new gambles, indicating that the functions’ shapes were meaningful. The monkeys were risk seeking (convex utility function) for low reward and risk avoiding (concave utility function) with higher amounts. Critically, the dopamine prediction error responses at the time of reward itself reflected the nonlinear utility functions measured at the time of choices. In particular, the reward response magnitude depended on the first derivative of the utility function and thus reflected the marginal utility. Furthermore, dopamine responses recorded outside of the task reflected the marginal utility of unpredicted reward. Accordingly, these responses were sufficient to train reinforcement learning models to predict the behaviorally defined expected utility of gambles. Conclusions These data suggest a neuronal manifestation of marginal utility in dopamine neurons and indicate a common neuronal basis for fundamental explanatory constructs in animal learning theory (prediction error) and economic decision theory (marginal utility).
    Current Biology 11/2014; 24:1-10. DOI:10.1016/j.cub.2014.08.064 · 9.92 Impact Factor
  • Martin O'Neill, Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Risk is a ubiquitous feature of the environment for all organisms. Very few things in life are achieved with absolute certainty. Therefore, it is essential that organisms process risky information efficiently to promote adaptive behaviour and enhance survival. Here we outline a clear definition of economic risk derived from economic theory and focus on two experiments in which we have shown subpopulations of single neurons in the orbitofrontal cortex of rhesus macaques that code either economic risk per se or an error-related risk signal, namely a risk prediction error. These biological risk signals are essential for processing and updating risky information in the environment to contribute to efficient decision making and adaptive behaviour.
    Journal of Physiology-Paris 06/2014; 109(1-3). DOI:10.1016/j.jphysparis.2014.06.002 · 2.35 Impact Factor
  • Source
    Armin Lak, William R Stauffer, Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Prediction error signals enable us to learn through experience. These experiences include economic choices between different rewards that vary along multiple dimensions. Therefore, an ideal way to reinforce economic choice is to encode a prediction error that reflects the subjective value integrated across these reward dimensions. Previous studies demonstrated that dopamine prediction error responses reflect the value of singular reward attributes that include magnitude, probability, and delay. Obviously, preferences between rewards that vary along one dimension are completely determined by the manipulated variable. However, it is unknown whether dopamine prediction error responses reflect the subjective value integrated from different reward dimensions. Here, we measured the preferences between rewards that varied along multiple dimensions, and as such could not be ranked according to objective metrics. Monkeys chose between rewards that differed in amount, risk, and type. Because their choices were complete and transitive, the monkeys chose "as if" they integrated different rewards and attributes into a common scale of value. The prediction error responses of single dopamine neurons reflected the integrated subjective value inferred from the choices, rather than the singular reward attributes. Specifically, amount, risk, and reward type modulated dopamine responses exactly to the extent that they influenced economic choices, even when rewards were vastly different, such as liquid and food. This prediction error response could provide a direct updating signal for economic values.
    Proceedings of the National Academy of Sciences 01/2014; 111(6). DOI:10.1073/pnas.1321596111 · 9.81 Impact Factor
  • Maria A Bermudez, Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Sensitivity to time, including the time of reward, guides the behaviour of all organisms. Recent research suggests that all major reward structures of the brain process the time of reward occurrence, including midbrain dopamine neurons, striatum, frontal cortex and amygdala. Neuronal reward responses in dopamine neurons, striatum and frontal cortex show temporal discounting of reward value. The prediction error signal of dopamine neurons includes the predicted time of rewards. Neurons in the striatum, frontal cortex and amygdala show responses to reward delivery and activities anticipating rewards that are sensitive to the predicted time of reward and the instantaneous reward probability. Together these data suggest that internal timing processes have several well characterized effects on neuronal reward processing.
    Philosophical Transactions of The Royal Society B Biological Sciences 01/2014; 369(1637):20120468. DOI:10.1098/rstb.2012.0468 · 6.31 Impact Factor
  • Martin O’Neill, Wolfram Schultz
    Journal of Physiology-Paris 01/2014; · 2.35 Impact Factor
  • Source
    Shunsuke Kobayashi, Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Basic tenets of sensory processing emphasize the importance of accurate identification and discrimination of environmental objects [1]. Although this principle holds also for reward, the crucial acquisition of reward for survival would be aided by the capacity to detect objects whose rewarding properties may not be immediately apparent. Animal learning theory conceptualizes how unrewarded stimuli induce behavioral reactions in rewarded contexts due to pseudoconditioning and higher-order context conditioning [2-6]. We hypothesized that the underlying mechanisms may involve context-sensitive reward neurons. We studied short-latency activations of dopamine neurons to unrewarded, physically salient stimuli while systematically changing reward context. Dopamine neurons showed substantial activations to unrewarded stimuli and their conditioned stimuli in highly rewarded contexts. The activations decreased and often disappeared entirely with stepwise separation from rewarded contexts. The influence of reward context suggests that dopamine neurons respond to real and potential reward. The influence of reward context is compatible with the reward nature of phasic dopamine responses. The responses may facilitate rapid, default initiation of behavioral reactions in environments usually containing reward. Agents would encounter more and miss less reward, resulting in survival advantage and enhanced evolutionary fitness.
    Current biology: CB 12/2013; 24(1). DOI:10.1016/j.cub.2013.10.061 · 9.92 Impact Factor
  • Source
    Raymundo Báez-Mendoza, Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Where and how does the brain code reward during social behavior? Almost all elements of the brain's reward circuit are modulated during social behavior. The striatum in particular is activated by rewards in social situations. However, its role in social behavior is still poorly understood. Here, we attempt to review its participation in social behaviors of different species ranging from voles to humans. Human fMRI experiments show that the striatum is reliably active in relation to others' rewards, to reward inequity and also while learning about social agents. Social contact and rearing conditions have long-lasting effects on behavior, striatal anatomy and physiology in rodents and primates. The striatum also plays a critical role in pair-bond formation and maintenance in monogamous voles. We review recent findings from single neuron recordings showing that the striatum contains cells that link own reward to self or others' actions. These signals might be used to solve the agency-credit assignment problem: the question of whose action was responsible for the reward. Activity in the striatum has been hypothesized to integrate actions with rewards. The picture that emerges from this review is that the striatum is a general-purpose subcortical region capable of integrating social information into coding of social action and reward.
    Frontiers in Neuroscience 12/2013; 7:233. DOI:10.3389/fnins.2013.00233
  • [Show abstract] [Hide abstract]
    ABSTRACT: Mental imagery refers to percept-like experiences in the absence of sensory input. Brain imaging studies suggest common, modality-specific, neural correlates imagery and perception. We associated abstract visual stimuli with either visually presented or imagined monetary rewards and scrambled pictures. Brain images for a group of 12 participants were collected using functional magnetic resonance imaging. Statistical analysis showed that human midbrain regions were activated irrespective of the monetary rewards being imagined or visually present. A support vector machine trained on the midbrain activation patterns to the visually presented rewards predicted with 75% accuracy whether the participants imagined the monetary reward or the scrambled picture during imagination trials. Training samples were drawn from visually presented trials and classification accuracy was assessed for imagination trials. These results suggest the use of machine learning technique for classification of underlying cognitive states from brain imaging data.
    2013 Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG); 12/2013
  • Martin O'Neill, Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Risk is a ubiquitous feature of life. It plays an important role in economic decisions by affecting subjective reward value. Informed decisions require accurate risk information for each choice option. However, risk is often not constant but changes dynamically in the environment. Therefore, risk information should be updated to the current risk level. Potential mechanisms involve error-driven updating, whereby differences between current and predicted risk levels (risk prediction errors) are used to obtain currently accurate risk predictions. As a major reward structure, the orbitofrontal cortex is involved in coding key reward parameters such as reward value and risk. In this study, monkeys viewed different visual stimuli indicating specific levels of risk that deviated from the overall risk predicted by a common earlier stimulus. A group of orbitofrontal neurons displayed a risk signal that tracked the discrepancy between current and predicted risk. Such neuronal signals may be involved in the updating of risk information.
    The Journal of Neuroscience : The Official Journal of the Society for Neuroscience 10/2013; 33(40):15810-15814. DOI:10.1523/JNEUROSCI.4236-12.2013 · 6.75 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Social interactions provide agents with the opportunity to earn higher benefits than when acting alone and contribute to evolutionary stable strategies. A basic requirement for engaging in beneficial social interactions is to recognize the actor whose movement results in reward. Despite the recent interest in the neural basis of social interactions, the neurophysiological mechanisms identifying the actor in social reward situations are unknown. A brain structure well suited for exploring this issue is the striatum, which plays a role in movement, reward, and goal-directed behavior. In humans, the striatum is involved in social processes related to reward inequity, donations to charity, and observational learning. We studied the neurophysiology of social action for reward in rhesus monkeys performing a reward-giving task. The behavioral data showed that the animals distinguished between their own and the conspecific's reward and knew which individual acted. Striatal neurons coded primarily own reward but rarely other's reward. Importantly, the activations occurred preferentially, and in approximately similar fractions, when either the own or the conspecific's action was followed by own reward. Other striatal neurons showed social action coding without reward. Some of the social action coding disappeared when the conspecific's role was simulated by a computer, confirming a social rather than observational relationship. These findings demonstrate a role of striatal neurons in identifying the social actor and own reward in a social setting. These processes may provide basic building blocks underlying the brain's function in social interactions.
    Proceedings of the National Academy of Sciences 09/2013; DOI:10.1073/pnas.1211342110 · 9.81 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: To make adaptive choices, humans need to estimate the probability of future events. Based on a Bayesian approach, it is assumed that probabilities are inferred by combining a priori, potentially subjective, knowledge with factual observations, but the precise neurobiological mechanism remains unknown. Here, we study whether neural encoding centers on subjective posterior probabilities, and data merely lead to updates of posteriors, or whether objective data are encoded separately alongside subjective knowledge. During fMRI, young adults acquired prior knowledge regarding uncertain events, repeatedly observed evidence in the form of stimuli, and estimated event probabilities. Participants combined prior knowledge with factual evidence using Bayesian principles. Expected reward inferred from prior knowledge was encoded in striatum. BOLD response in specific nodes of the default mode network (angular gyri, posterior cingulate, and medial prefrontal cortex) encoded the actual frequency of stimuli, unaffected by prior knowledge. In this network, activity increased with frequencies and thus reflected the accumulation of evidence. In contrast, Bayesian posterior probabilities, computed from prior knowledge and stimulus frequencies, were encoded in bilateral inferior frontal gyrus. Here activity increased for improbable events and thus signaled the violation of Bayesian predictions. Thus, subjective beliefs and stimulus frequencies were encoded in separate cortical regions. The advantage of such a separation is that objective evidence can be recombined with newly acquired knowledge when a reinterpretation of the evidence is called for. Overall this study reveals the coexistence in the brain of an experience-based system of inference and a knowledge-based system of inference.
    The Journal of Neuroscience : The Official Journal of the Society for Neuroscience 06/2013; 33(26):10887-10897. DOI:10.1523/JNEUROSCI.5829-12.2013 · 6.75 Impact Factor
  • Source
  • Source
    Wolfram Schultz
    [Show abstract] [Hide abstract]
    ABSTRACT: Recent work has advanced our knowledge of phasic dopamine reward prediction error signals. The error signal is bidirectional, reflects well the higher order prediction error described by temporal difference learning models, is compatible with model-free and model-based reinforcement learning, reports the subjective rather than physical reward value during temporal discounting and reflects subjective stimulus perception rather than physical stimulus aspects. Dopamine activations are primarily driven by reward, and to some extent risk, whereas punishment and salience have only limited activating effects when appropriate controls are respected. The signal is homogeneous in terms of time course but heterogeneous in many other aspects. It is essential for synaptic plasticity and a range of behavioural learning situations.
    Current opinion in neurobiology 12/2012; 23(2). DOI:10.1016/j.conb.2012.11.012 · 6.77 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The amygdala is a key structure of the brain's reward system. Existing theories view its role in decision-making as restricted to an early valuation stage that provides input to decision mechanisms in downstream brain structures. However, the extent to which the amygdala itself codes information about economic choices is unclear. Here, we report that individual neurons in the primate amygdala predict behavioral choices in an economic decision task. We recorded the activity of amygdala neurons while monkeys chose between saving liquid reward with interest and spending the accumulated reward. In addition to known value-related responses, we found that activity in a group of amygdala neurons predicted the monkeys' upcoming save-spend choices with an average accuracy of 78%. This choice-predictive activity occurred early in trials, even before information about specific actions associated with save-spend choices was available. For a substantial number of neurons, choice-differential activity was specific for free, internally generated economic choices and not observed in a control task involving forced imperative choices. A subgroup of choice-predictive neurons did not show relationships to value, movement direction, or visual stimulus features. Choice-predictive activity in some amygdala neurons was preceded by transient periods of value coding, suggesting value-to-choice transitions and resembling decision processes in other brain systems. These findings suggest that the amygdala might play an active role in economic decisions. Current views of amygdala function should be extended to incorporate a role in decision-making beyond valuation.
    Proceedings of the National Academy of Sciences 10/2012; 109(46). DOI:10.1073/pnas.1212706109 · 9.81 Impact Factor

Publication Stats

26k Citations
985.18 Total Impact Points

Institutions

  • 2003–2015
    • University of Cambridge
      • Department of Physiology, Development and Neuroscience
      Cambridge, England, United Kingdom
  • 2008
    • Stanford University
      • Department of Neurobiology
      Stanford, CA, United States
  • 1981–2005
    • Université de Fribourg
      • Institut de la Pathologie
      Freiburg, Fribourg, Switzerland
  • 1978
    • Karolinska Institutet
      Solna, Stockholm, Sweden