Figure - available from: Nature Communications
This content is subject to copyright. Terms and conditions apply.
Neural coding of probability and magnitude of rewards in the four brain regions a Illustration of neural recording areas based on coronal magnetic resonance images. b Example activity histogram of a DS neuron modulated by probability and magnitude of rewards with positive regression coefficients during the single-cue task (P + M + type). The activity aligned to the cue onset is represented for three different levels of probability (0.1–0.3, 0.4–0.7, and 0.8–1.0) and magnitude (0.1–0.3 mL, 0.4–0.7 mL, and 0.8–1.0 mL) of rewards. Gray hatched time windows indicate the 1-s time window used to estimate the neural firing rates shown in f and g. Raster grams are shown below. c–e similar to b, but for VS, cOFC, and mOFC neurons. f Plot of the neural firing rates during the 1-s time window in b for ten levels of probability and magnitude of rewards. The firings are normalized by the maximum firing rates. P and M indicate the probability and magnitude of rewards, respectively. g Color map of the neural firing rates during the 1 s time window in b for ten levels of probability and magnitude of rewards. Average smoothing was made between neighboring pixels. h Percentage of neurons modulated by probability and magnitude of rewards in the four core reward brain regions. Gray indicates activity showing positive regression coefficients for probability and magnitude of rewards (P + M + type). Black indicates activity showing the negative regression coefficients for probability and magnitude (P-M- type). Images in panels a were created by the authors and previously published in Neural Population Dynamics Underlying Expected Value Computation. Hiroshi Yamada, et al.²⁸. https://creativecommons.org/licenses/by/4.0/.
Source publication
Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in fo...
Citations
... In general, we examine the outcome of our choice and adjust subsequent choice behavior using the outcome information to choose an appropriate action. Five significant studies on neurons (Kawai et al. (2015); Yamada et al. (2021); Imaizumi et al. (2022); Yang et al. (2022); Ferrari-Toniolo and Schultz (2023)) have examined neuronal responses to loss and gain. These studies suggest that two different neural systems may respond to loss and gain, resulting in a value function with a cusp as a reference point. ...
... In their studies on brain reward circuitry, Yamada et al. (2021) and Imaizumi et al. (2022) proposed a neuronal prospect theory model. Using theoretical accuracy equivalent to that of human neuroimaging studies from a gain perspective, they showed that single-neuron activity in four core reward-related cortical and subcortical regions represents a subjective assessment of risky gambles in monkeys. ...
... In the gain context, both monkeys were risk-seekers when the starting token number was low; however, both demonstrated risk-neutral or risk-averse behavior when the start token number increased. This result is consistent with Yamada et al. (2021) and Imaizumi et al. (2022). Moreover, Yang et al. (2022) showed, in monkey G, the utility functions (value functions) in monkey G were consistently steeper for losses than for gains. ...
In prospect theory, the value function is typically concave for gains and convex for losses, with losses usually having a steeper slope than gains. The neural system responds differently to losses and gains. Five new studies on neurons related to this issue have examined neuronal responses to losses, gains, and reference points. This study investigated a new concept of the value function. A value function with a neuronal cusp may exhibit variations and behavioral cusps associated with catastrophic events, potentially influencing a trader's decision to close a position. Additionally, we have conducted empirical studies on algorithmic trading strategies that employ different value function specifications.
... This phenomenon results in a concave probability weighting function and constitutes a key component of prospect theory, a fundamental theory in economics [4,10]. Studies have been done to explore the neural mechanisms underlying such probability distortion when outcome probabilities are explicitly shown to the subjects [11,14,15]. ...
Making decisions when outcomes are uncertain requires accurate judgment of the probability of outcomes, yet such judgments are often inaccurate, owing to reliance on heuristics that introduce systematic errors like overweighting of low probabilities. Here, using a decision-making task in which the participants were unaware of outcome probabilities, we discovered that both humans and mice exhibit a rarity-induced decision bias (RIDB), i.e., a preference towards rare rewards, which persists across task performance. Optogenetics experiments demonstrated that activity in the posterior parietal cortex (PPC) is required for the RIDB. Using in vivo electrophysiology, we found that rare rewards bidirectionally modulate choice-encoding PPC neurons to bias subsequent decisions towards rare rewards. Learning enhances stimulus-encoding of PPC neurons, which plays a causal role in stimulus-guided decisions. We then developed a dual-agent behavioural model that successfully recapitulates the decision-making and learning behaviours, and corroborates the specific functions of PPC neurons in mediating decision-making and learning. Thus, beyond expanding understanding of rare probability overweighting to a context where the outcome probability is unknown, and characterizing the neural basis for RIDB in the PPC, our study reveals an evolutionarily conserved heuristic that persistently impacts decision-making and learning under uncertainty.
... This aforementioned limitation introduces several potential problems in neuroeconomic studies (Camerer et al., 2005;Glimcher et al., 2008;Yamada et al., 2021;Imaizumi et al., 2022;Tymula et al., 2023) that employ experimental testing of reward valuation systems for economic choices. When measuring neural activity in the reward circuitry, the subjective values of any reward depend on the physical state of the subject (Nakano et al., 1984;Critchley and Rolls, 1996;de Araujo et al., 2006;Pritchard et al., 2008), even for money (Symmonds et al., 2010). ...
Hunger and thirst drive animals’ consumption behavior and regulate their decision-making concerning rewards. We previously assessed the thirst states of monkeys by measuring blood osmolality under controlled water access and examined how these thirst states influenced their risk-taking behavior in decisions involving fluid rewards. However, hunger assessment in monkeys remains poorly performed. Moreover, the lack of precise measures for hunger states leads to another issue regarding how hunger and thirst states interact with each other in each individual. Thus, when controlling food access to motivate performance, it remains unclear how these two physiological needs are satisfied in captive monkeys. Here, we measured blood ghrelin and osmolality levels to respectively assess hunger and thirst in four captive macaques. Using an enzyme-linked immunosorbent assay, we identified that the levels of blood ghrelin, a widely measured hunger-related peptide hormone in humans, were high after 20 h of no food access (with ad libitum water). This reflects a typical controlled food access condition. One hour after consuming a regular dry meal, the blood ghrelin levels in three out of four monkeys decreased to within their baseline range. Additionally, blood osmolality measured from the same blood sample, the standard hematological index of hydration status, increased after consuming the regular dry meal with no water access. Thus, ghrelin and osmolality may reflect the physiological states of individual monkeys regarding hunger and thirst, suggesting that these indices can be used as tools for monitoring hunger and thirst levels that mediate an animal's decision to consume rewards.
... To choose an appropriate action, we monitor the outcome of our choice and adjust subsequent choice behavior using the outcome information. Four important studies on neurons related to this issue have recently been published (Kawai et al. [2015], Yamada et al. [2021], Imaizumi et al. [2022], Yang et al. [2022]). These studies examine neuronal responses to loss and gain, respectively. ...
... Yamada et al. [2021] and Imaizumi et al. [2022] propose a neuronal prospect theory model in the brain reward circuitry. They showed, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys on gain side. ...
... In the gain context, both monkeys were risk-seeking when the start token number was low, but they became risk neutral or risk-averse when the start token number increased. This result is consistent with Yamada et al. [2021], Imaizumi et al. [2022]. Moreover, Yang et al. [2022] showed, in monkey G, the utility functions arXiv A PREPRINT (value functions) were consistently steeper for losses than for gains. ...
In the prospect theory, value function is typically concave for gains, commonly convex for losses, with losses usually having a steeper slope than gains. The neural system largely differs from the loss and gains sides. Five new studies on neurons related to this issue have examined neuronal responses to losses, gains, and reference points. This study investigates a new concept of the value function. A value function with a neuronal cusp may show variations and behavior cusps with catastrophe where a trader closes one's position.
... www.nature.com/scientificreports/ In summary, the correlation for manufacturing capacity sharing is mostly based on two subjects making decisions, and most of them are only limited to evolutionary game theory, while ignoring the perceptual differences between people in the decision-making process 21 . What are the strategic choices of each party involved in the sharing of manufacturing capacity? ...
In order to investigate the strategy choice of each player in capacity sharing, the article constructs a tripartite game model based on capacity provider-capacity demander-government, introduces the prospect theory and conducts numerical simulation analysis using MATLAB. The results show that capacity sharing in the manufacturing industry is related to three parties: capacity providers, capacity demanders and the government, and their strategies in the game process influence each other; the sensitivity of capacity providers and capacity demanders is higher than that of the government; the increase of risk-return coefficient and loss-avoidance coefficient is conducive to the evolution of subjects to the ideal state.
... In our previous study, fluctuating neural population signals were observed in the dorsal striatum (DS) and medial OFC (mOFC) because of signal instability or weakness (Yamada et al., 2021, their Fig. 5A,B;Imaizumi et al., 2022). As the signal carried by the mOFC population was weak (Yamada et al., 2021, their Fig. 8, bottom row), eigenvector fluctuation in the mOFC population reflected the weak signal modulations by the probability and magnitude of rewards. ...
Neural population dynamics provide a key computational framework for understanding information processing in the sensory, cognitive, and motor functions of the brain. They systematically depict complex neural population activity, dominated by strong temporal dynamics as trajectory geometry in a low-dimensional neural space. However, neural population dynamics are poorly related to the conventional analytical framework of single-neuron activity, the rate-coding regime that analyzes firing rate modulations using task parameters. To link the rate-coding and dynamic models, we developed a variant of state-space analysis in the regression subspace, which describes the temporal structures of neural modulations using continuous and categorical task parameters. In macaque monkeys, using two neural population datasets containing either of two standard task parameters, continuous and categorical, we revealed that neural modulation structures are reliably captured by these task parameters in the regression subspace as trajectory geometry in a lower dimension. Furthermore, we combined the classical optimal-stimulus response analysis (usually used in rate-coding analysis) with the dynamic model and found that the most prominent modulation dynamics in the lower dimension were derived from these optimal responses. Using those analyses, we successfully extracted geometries for both task parameters that formed a straight geometry, suggesting that their functional relevance is characterized as a unidimensional feature in their neural modulation dynamics. Collectively, our approach bridges neural modulation in the rate-coding model and the dynamic system, and provides researchers with a significant advantage in exploring the temporal structure of neural modulations for pre-existing datasets.
... Previous studies have implied that findings from monkey research expand our understanding of human behavior by enabling researchers to understand the evolutionary roots of choice, as well as to conduct research that is not easily feasible with human participants. For example, we have previously described utility and probability weighting functions in monkeys using the same dataset in the present study (24). Leveraging the use of the same experimental paradigm for both species enabled us to compare monkey and human behavior through the lens of two major decision theories and to contribute to the long-standing dispute on the extent to which monkeys are a good model for human decision-making. ...
... Our previous publications (24,26) reported estimates of the expected value model and static prospect theory models for risky choices of the monkeys using the same datasets in this study. We have never reported human experimental results or a dynamic prospect theory model. ...
... After a positive RPE, both monkeys and humans significantly overweighted all probabilities (Fig. 4B, solid orange line) more than after no RPE (Fig. 4B, gray line), suggesting that both species share a common valuation: greater optimism regarding the probability of winning after experiencing a larger positive RPE. After a positive RPE, dopamine is released from the synaptic terminal (19,20) to guide learning and form predictions (24) for the next decision. This mechanism may drive animals to maximize utility in a broader decision-making context when the environment is less stable by learning predictions in the target brain regions (38)(39)(40). ...
Research in the multidisciplinary field of neuroeconomics has mainly been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-making in a comprehensive manner. Here, we propose and test a decision-making theory under uncertainty that combines these highly influential theories. Collecting many gambling decisions from laboratory monkeys allowed for reliable testing of our model and revealed a systematic violation of prospect theory's assumption that probability weighting is static. Using the same experimental paradigm in humans, substantial similarities between these species were uncovered by various econometric analyses of our dynamic prospect theory model, which incorporates decision-by-decision learning dynamics of prediction errors into static prospect theory. Our model provides a unified theoretical framework for exploring a neurobiological model of economic choice in human and nonhuman primates.
Neural dynamics are thought to reflect computations that relay and transform information in the brain. Previous studies have identified the neural population dynamics in many individual brain regions as a trajectory geometry, preserving a common computational motif. However, whether these populations share particular geometric patterns across brain-wide neural populations remains unclear. Here, by mapping neural dynamics widely across temporal/frontal/limbic regions in the cortical and subcortical structures of monkeys, we show that 10 neural populations, including 2,500 neurons, propagate visual item information in a stochastic manner. We found that visual inputs predominantly evoked rotational dynamics in the higher-order visual area, TE, and its downstream striatum tail, while curvy/straight dynamics appeared frequently downstream in the orbitofrontal/hippocampal network. These geometric changes were not deterministic but rather stochastic according to their respective emergence rates. Our meta-analysis results indicate that visual information propagates as a heterogeneous mixture of stochastic neural population signals in the brain.
Behavior-related neuronal signals often vary between neurons, which might reflect the unreliability of individual neurons or a truly heterogeneous code. This notion may also apply to economic ("value-based") choices and the underlying reward signals. Reward value is subjective and can be described by a nonlinearly weighted magnitude (utility) and probability. Defining subjective values relies on the continuity axiom, whose testing involves structured variations of a wide range of reward magnitudes and probabilities. Axiom compliance demonstrates understanding of the stimuli and the meaningful character of choices. Using these tests, we investigated the encoding of subjective economic value by neurons in a key economic-decision structure of the monkey brain, the orbitofrontal cortex (OFC). We found that individual neurons carry heterogeneous neuronal value signals that largely fail to match the animal's choices. However, neuronal population signals matched the animal's choices well, suggesting accurate subjective economic value encoding by a heterogeneous population of unreliable neurons.
