Yuri Imaizumi’s research while affiliated with University of Tsukuba and other places

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals’ choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

Research in the multidisciplinary field of neuroeconomics has been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-making in a comprehensive manner. Here, we propose and test a new decision-making theory under uncertainty that combines these highly influential theories. Collecting many gambling decisions from laboratory monkeys allowed for reliable testing of our hybrid model and revealed a systematic violation of prospect theory’s assumption that probability weighting is static. Using the same experimental paradigm in humans, substantial similarities between monkey and human behavior were described by our hybrid model, which incorporates decision-by-decision learning dynamics of prediction errors into static prospect theory. Our new model provides a single unified theoretical framework for exploring the neurobiological model of economic choice in human and nonhuman primates.

Neural population dynamics, presumably fundamental computational units in the brain, provide a key framework for understanding information processing in the sensory, cognitive, and motor functions. However, neural population dynamics is not explicitly related to the conventional analytic framework for single-neuron activity, i.e., representational models that analyze neuronal modulations associated with cognitive and motor parameters. In this study, we applied a recently developed state-space analysis to incorporate the representational models into the dynamic model in combination with these parameters. We compared neural population dynamics between continuous and categorical task parameters during two visual recognition tasks, using the datasets originally designed for a single-neuron approach. We successfully extracted neural population dynamics in the regression subspace, which represent modulation dynamics for both continuous and categorical task parameters with reasonable temporal characteristics. Furthermore, we combined the classical optimal-stimulus analysis paradigm for the single-neuron approach (i.e., stimulus identified as maximum neural responses) into the dynamic model, and found that the most prominent modulation dynamics at the lower dimension were derived from these optimal responses. Thus, our approach provides a unified framework for incorporating knowledge acquired with the single-neuron approach into the dynamic model as a standard procedure for describing neural modulation dynamics in the brain.

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, reflects prospect theory remains unknown. Here, we show with theoretical accuracy equivalent to that of human neuroimaging studies that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards, parameterized as a multiplicative combination of a utility and probability weighting functions in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reliably reconstructed risk preferences and subjective probability perceptions revealed by the animals' choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

Research in behavioral economics and reinforcement learning has given rise to two influential theories describing human economic choice under uncertainty. The first, prospect theory, assumes that decision-makers use static mathematical functions, utility and probability weighting, to calculate the values of alternatives. The second, reinforcement learning theory, posits that dynamic mathematical functions update the values of alternatives based on experience through reward prediction error (RPE). To date, these theories have been examined in isolation without reference to one another. Therefore, it remains unclear whether RPE affects a decision-maker’s utility and/or probability weighting functions, or whether these functions are indeed static as in prospect theory. Here, we propose a dynamic prospect theory model that combines prospect theory and RPE, and test this combined model using choice data on gambling behavior of captive macaques. We found that under standard prospect theory, monkeys, like humans, had a concave utility function. Unlike humans, monkeys exhibited a concave, rather than inverse-S shaped, probability weighting function. Our dynamic prospect theory model revealed that probability distortions, not the utility of rewards, solely and systematically varied with RPE: after a positive RPE, the estimated probability weighting functions became more concave, suggesting more optimistic belief about receiving rewards and over-weighted subjective probabilities at all probability levels. Thus, the probability perceptions in laboratory monkeys are not static even after extensive training, and are governed by a dynamic function well captured by the algorithmic feature of reinforcement learning. This novel evidence supports combining these two major theories to capture choice behavior under uncertainty. Significance statement We propose and test a new decision theory under uncertainty by combining pre-existing two influential theories in the neuroeconomics: prospect theory from economics and prediction error theory from reinforcement learning. Collecting a large dataset (over 60,000 gambling decisions) from laboratory monkeys enables us to test the hybrid model of these two core decision theories reliably. Our results showed over-weighted subjective probabilities at all probability levels after lucky win, indicating that positive prediction error systematically bias decision-makers more optimistically about receiving rewards. This trial-by-trial prediction-error dynamics in probability perception provides outperformed performance of the model compared to the standard static prospect theory. Thus, both static and dynamic elements coexist in monkey’s risky decision-making, an evidence contradicting the assumption of prospect theory.

Computation of expected values (i.e., probability × magnitude) seems to be a dynamic integrative process performed by the brain for efficient economic behavior. However, neural dynamics underlying this computation is largely unknown. Using lottery tasks in monkeys ( Macaca mulatta , male; Macaca fuscata , female), we examined (1) whether four core reward-related brain regions detect and integrate probability and magnitude cued by numerical symbols and (2) whether these brain regions have distinct dynamics in the integrative process. Extraction of the mechanistic structure of neural population signals demonstrated that expected value signals simultaneously arose in the central orbitofrontal cortex (cOFC; medial part of area 13) and ventral striatum (VS). Moreover, these signals were incredibly stable compared with weak and/or fluctuating signals in the dorsal striatum and medial OFC. Temporal dynamics of these stable expected value signals were unambiguously distinct: sharp and gradual signal evolutions in the cOFC and VS, respectively. These intimate dynamics suggest that the cOFC and VS compute the expected values with unique time constants, as distinct, partially overlapping processes. SIGNIFICANCE STATEMENT Our results differ from those of earlier studies suggesting that many reward-related regions in the brain signal probability and/or magnitude and provide a mechanistic structure for expected value computation employed in multiple neural populations. A central part of the orbitofrontal cortex (cOFC) and ventral striatum (VS) can simultaneously detect and integrate probability and magnitude into an expected value. Our empirical study on these neural population dynamics raises a possibility that the cOFC and VS cooperate on this computation with unique time constants as distinct, partially overlapping processes.