Fig 4 - uploaded by Hiroshi Yamada
Content may be subject to copyright.
The effect of the RPE through utility and probability weighting functions in the dynamic prospect theory model. Effect of RPE on the utility function (A) and probability weighting function (B) for monkey SUN (top), monkey FU (middle), and human participants (bottom). All curves were drawn using the estimates from Table 4 (model 2). Solid orange lines are drawn assuming the largest RPE (0.9 for monkeys and 4.5 for humans), dashed orange lines are drawn assuming the smallest RPE (−0.9 for monkeys and −4.5 for humans), and the dotted gray line represents RPE = 0. Note that in humans, the dotted gray and dashed orange curves overlap in (A). For illustrative purposes, we limited the y axis and x axis to 0 and 1 for humans to allow for an easier comparison with monkeys' utility functions.
Source publication
Research in the multidisciplinary field of neuroeconomics has mainly been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-ma...
Contexts in source publication
Context 1
... of the probability weighting function parameter delta on positive RPE in both monkeys and humans (Table 4; see delta parameters, Δpositive t−1 in model 2). The significant negative coefficients of delta, Δpositive t−1 , observed in all three datasets indicate that the larger the positive RPE, the more concave the probability weighting function (Fig. 4B, compare solid orange and gray dotted curves). In contrast, no significant effect was observed in the gamma parameter (Table 4, see ...
Context 2
... effect of a more optimistic perception of reward probability after an unexpected positive outcome, such as good fortune or winning a jackpot. The utility function governed by the alpha parameter was not consistently affected by RPE, as shown by no significant effects of positive and negative RPEs in most cases (Table 4, see alpha in model 2; see Fig. 4A). In model 1, the curvature of the utility function was affected by the overall RPE in monkey SUN and human participants, but with the opposite effects (Table 4; see alpha in model 1). Thus, it is likely that the RPE does not consistently affect the utility curvature in the same way, in contrast to the consistent effect of a positive ...
Context 3
... RPE and updated their subjective valuations of the gambles through experience. Consistently, human participants were sensitive to RPE, although they made decisions based on known probabilities and magnitudes of rewards, presented using visual pie charts. After a positive RPE, both monkeys and humans significantly overweighted all probabilities (Fig. 4B, solid orange line) more than after no RPE (Fig. 4B, gray line), suggesting that both species share a common valuation: greater optimism regarding the probability of winning after experiencing a larger positive RPE. After a positive RPE, dopamine is released from the synaptic terminal (19,20) to guide learning and form predictions (24) ...
Context 4
... gambles through experience. Consistently, human participants were sensitive to RPE, although they made decisions based on known probabilities and magnitudes of rewards, presented using visual pie charts. After a positive RPE, both monkeys and humans significantly overweighted all probabilities (Fig. 4B, solid orange line) more than after no RPE (Fig. 4B, gray line), suggesting that both species share a common valuation: greater optimism regarding the probability of winning after experiencing a larger positive RPE. After a positive RPE, dopamine is released from the synaptic terminal (19,20) to guide learning and form predictions (24) for the next decision. This mechanism may drive animals to ...
Context 5
... we showed that the assumption in prospect theory that the probability weighting function is static would not be sufficient to accurately describe the behavior of macaques and humans (Fig. 4B and Table 5). Our combined model indicated that both monkeys and humans overweighed all probabilities significantly more when the positive RPE was larger compared to no RPE. The observation that probability weighting is not static was supported by earlier findings in the monkey literature, where the same animals exhibited different ...
Context 6
... was the improvement in the model fit among the sequentially developed models (Fig. 2A). In humans, the EU largely improved the explanatory ability, while the two-parameter probability weighting functions improved it in monkeys. Moreover, the effect of negative RPEs may also differ between these two species, but this difference was inconsistent (Fig. 4B), possibly because the sensitivity to negative events varied between individuals (34,35). Together, these two studies reinforce the view that methodology across studies must be aligned for crossspecies comparisons. This calls for further studies in which different species are studied under the same ...
Context 7
... of the probability weighting function parameter delta on positive RPE in both monkeys and humans (Table 4; see delta parameters, Δpositive t−1 in model 2). The significant negative coefficients of delta, Δpositive t−1 , observed in all three datasets indicate that the larger the positive RPE, the more concave the probability weighting function (Fig. 4B, compare solid orange and gray dotted curves). In contrast, no significant effect was observed in the gamma parameter (Table 4, see ...
Context 8
... effect of a more optimistic perception of reward probability after an unexpected positive outcome, such as good fortune or winning a jackpot. The utility function governed by the alpha parameter was not consistently affected by RPE, as shown by no significant effects of positive and negative RPEs in most cases (Table 4, see alpha in model 2; see Fig. 4A). In model 1, the curvature of the utility function was affected by the overall RPE in monkey SUN and human participants, but with the opposite effects (Table 4; see alpha in model 1). Thus, it is likely that the RPE does not consistently affect the utility curvature in the same way, in contrast to the consistent effect of a positive ...
Context 9
... RPE and updated their subjective valuations of the gambles through experience. Consistently, human participants were sensitive to RPE, although they made decisions based on known probabilities and magnitudes of rewards, presented using visual pie charts. After a positive RPE, both monkeys and humans significantly overweighted all probabilities (Fig. 4B, solid orange line) more than after no RPE (Fig. 4B, gray line), suggesting that both species share a common valuation: greater optimism regarding the probability of winning after experiencing a larger positive RPE. After a positive RPE, dopamine is released from the synaptic terminal (19,20) to guide learning and form predictions (24) ...
Context 10
... gambles through experience. Consistently, human participants were sensitive to RPE, although they made decisions based on known probabilities and magnitudes of rewards, presented using visual pie charts. After a positive RPE, both monkeys and humans significantly overweighted all probabilities (Fig. 4B, solid orange line) more than after no RPE (Fig. 4B, gray line), suggesting that both species share a common valuation: greater optimism regarding the probability of winning after experiencing a larger positive RPE. After a positive RPE, dopamine is released from the synaptic terminal (19,20) to guide learning and form predictions (24) for the next decision. This mechanism may drive animals to ...
Context 11
... we showed that the assumption in prospect theory that the probability weighting function is static would not be sufficient to accurately describe the behavior of macaques and humans (Fig. 4B and Table 5). Our combined model indicated that both monkeys and humans overweighed all probabilities significantly more when the positive RPE was larger compared to no RPE. The observation that probability weighting is not static was supported by earlier findings in the monkey literature, where the same animals exhibited different ...
Context 12
... was the improvement in the model fit among the sequentially developed models (Fig. 2A). In humans, the EU largely improved the explanatory ability, while the two-parameter probability weighting functions improved it in monkeys. Moreover, the effect of negative RPEs may also differ between these two species, but this difference was inconsistent (Fig. 4B), possibly because the sensitivity to negative events varied between individuals (34,35). Together, these two studies reinforce the view that methodology across studies must be aligned for crossspecies comparisons. This calls for further studies in which different species are studied under the same ...
Citations
... Our definition of CPT-value employs a kind of "static" semantics, in the sense that we fix a complete strategy for the whole MDP and then view it as a prospect. A "dynamic" semantics is also of interest, where the current state and the history that led up to it affect the decision; such a dynamic prospect theory has recently been proposed [53]. This comes with several complications and design choices to be made, as discussed in [1, App. ...
Cumulative prospect theory (CPT) is the first theory for decision-making under uncertainty that combines full theoretical soundness and empirically realistic features [P.P. Wakker - Prospect theory: For risk and ambiguity, Page 2]. While CPT was originally considered in one-shot settings for risk-aware decision-making, we consider CPT in sequential decision-making. The most fundamental and well-studied models for sequential decision-making are Markov chains (MCs), and their generalization Markov decision processes (MDPs). The complexity theoretic study of MCs and MDPs with CPT is a fundamental problem that has not been addressed in the literature. Our contributions are as follows: First, we present an alternative viewpoint for the CPT-value of MCs and MDPs. This allows us to establish a connection with multi-objective reachability analysis and conclude the strategy complexity result that memoryless randomized strategies are necessary and sufficient for optimality. Second, based on this connection, we provide an algorithm for computing the CPT-value in MDPs with infinite-horizon objectives. We show that the problem is in EXPTIME and fixed-parameter tractable. Moreover, we provide a polynomial-time algorithm for the special case of MCs.
... However, as captured in expected utility theory, decisionmakers are usually not indifferent; they have risk preferences. Tversky and Kahneman (1981) introduced these kinds of problems to illustrate critical tests of divergent predictions of expected utility theory versus prospect theory, still both widely used theories today (e.g., Barberis, 2013;Tymula et al., 2023). Prospect theory predicted gain-loss differences in risk preference, which was thought to rule out expected utility theory in its classic form. ...
Framing effects (risk preferences reverse for gains vs. losses) and the Allais paradox (risk preferences reverse when an option is certain vs. not) are major violations of rational choice theory. In contrast to typical samples, certified public accountants who are competent in working with probabilities and expected values should be an ideal test case for rational choice, especially high scorers on the cognitive reflection test (CRT). Although dual-process theories emphasize numeracy and cognitive reflection, fuzzy-trace theory emphasizes gist-based intuition to explain these effects among cognitively advanced decision-makers. Thus, we recruited a high-numeracy sample of certified public accountants (N = 259) and students (N = 648). We administered classic dread-disease framing, business framing, and Allais paradox problems and the CRT. Each participant received a gain and loss framing problem from different domains (one disease and one business), with presentation order counterbalanced across participants. Order of Allais problems was counterbalanced within participants. Within-participants (cross-domain) framing, between-participants (within-domain) framing, and the Allais paradox were observed for both samples. Accountants did not show domain-specific attenuation (differentially smaller framing) for business problems. Despite large expected-value differences between Allais problem options, accountants’ choices resembled students’ choices. Contrary to dual-process theories, CRT scores were positively related to framing for students (more framing with higher CRT) and inconsistently related for accountants, but high scorers had robust framing effects; high scorers also showed the Allais paradox. Results are consistent with fuzzy-trace theory’s expectation that experts show framing effects because they rely primarily on gist-based intuition, not because they lack numeracy or cognitive reflection.
... Schneider & Day, 2018;Tymula et al., 2023)。 鉴 于反 S 型概率权重的灵活性(Abdellaoui et al., 2010), ...
... This phenomenon results in a concave probability weighting function and constitutes a key component of prospect theory, a fundamental theory in economics [4,10]. Studies have been done to explore the neural mechanisms underlying such probability distortion when outcome probabilities are explicitly shown to the subjects [11,14,15]. ...
Making decisions when outcomes are uncertain requires accurate judgment of the probability of outcomes, yet such judgments are often inaccurate, owing to reliance on heuristics that introduce systematic errors like overweighting of low probabilities. Here, using a decision-making task in which the participants were unaware of outcome probabilities, we discovered that both humans and mice exhibit a rarity-induced decision bias (RIDB), i.e., a preference towards rare rewards, which persists across task performance. Optogenetics experiments demonstrated that activity in the posterior parietal cortex (PPC) is required for the RIDB. Using in vivo electrophysiology, we found that rare rewards bidirectionally modulate choice-encoding PPC neurons to bias subsequent decisions towards rare rewards. Learning enhances stimulus-encoding of PPC neurons, which plays a causal role in stimulus-guided decisions. We then developed a dual-agent behavioural model that successfully recapitulates the decision-making and learning behaviours, and corroborates the specific functions of PPC neurons in mediating decision-making and learning. Thus, beyond expanding understanding of rare probability overweighting to a context where the outcome probability is unknown, and characterizing the neural basis for RIDB in the PPC, our study reveals an evolutionarily conserved heuristic that persistently impacts decision-making and learning under uncertainty.
... This aforementioned limitation introduces several potential problems in neuroeconomic studies (Camerer et al., 2005;Glimcher et al., 2008;Yamada et al., 2021;Imaizumi et al., 2022;Tymula et al., 2023) that employ experimental testing of reward valuation systems for economic choices. When measuring neural activity in the reward circuitry, the subjective values of any reward depend on the physical state of the subject (Nakano et al., 1984;Critchley and Rolls, 1996;de Araujo et al., 2006;Pritchard et al., 2008), even for money (Symmonds et al., 2010). ...
Hunger and thirst drive animals’ consumption behavior and regulate their decision-making concerning rewards. We previously assessed the thirst states of monkeys by measuring blood osmolality under controlled water access and examined how these thirst states influenced their risk-taking behavior in decisions involving fluid rewards. However, hunger assessment in monkeys remains poorly performed. Moreover, the lack of precise measures for hunger states leads to another issue regarding how hunger and thirst states interact with each other in each individual. Thus, when controlling food access to motivate performance, it remains unclear how these two physiological needs are satisfied in captive monkeys. Here, we measured blood ghrelin and osmolality levels to respectively assess hunger and thirst in four captive macaques. Using an enzyme-linked immunosorbent assay, we identified that the levels of blood ghrelin, a widely measured hunger-related peptide hormone in humans, were high after 20 h of no food access (with ad libitum water). This reflects a typical controlled food access condition. One hour after consuming a regular dry meal, the blood ghrelin levels in three out of four monkeys decreased to within their baseline range. Additionally, blood osmolality measured from the same blood sample, the standard hematological index of hydration status, increased after consuming the regular dry meal with no water access. Thus, ghrelin and osmolality may reflect the physiological states of individual monkeys regarding hunger and thirst, suggesting that these indices can be used as tools for monitoring hunger and thirst levels that mediate an animal's decision to consume rewards.
... On the behavioral level logistic mixed-effects regression models were employed to determine the predictive capacity of emotional and reward PEs to punishment decisions (Fig. 1c). On the neural level, we aimed to investigate: 1) the common and distinct neural systems that support the prediction and experience of reward and emotion during social decisions, 2) whether distinct multivariate neural patterns are sensitive to capture variations of emotional and reward PEs and the associated punishment decision using machine-learning based neural decoding approaches (given the higher precision of this approach to establish process specific neural signatures 30,31 ), 3) whether the reward or emotional neurofunctional PE representation predict the neurofunctional decision to reject an offer and thus to characterize the different roles the PEs play in punishment decisions (Fig. 1c). The modified Ultimatum Game task and PEs computation. ...
Traditional decision-making models conceptualize humans as optimal learners aiming to maximize outcomes by leveraging reward prediction errors (PE). While violated emotional expectations(emotional PEs) have recently been formalized, the underlying neurofunctional basis and whether it differs from reward PEs remain unclear. Using a modified fMRI Ultimatum Game on n=43 participants we modelled reward and emotional PEs in response to unfair offers and subsequent punishment decisions. Computational modelling revealed distinct contributions of reward and emotional PEs to
punishment decisions, with reward PE exerting a stronger impact. This process was neurofunctionally dissociable such that (1) reward engaged the dorsomedial prefrontal cortex while emotional experience recruited the anterior insula, (2) multivariate decoding accurately separated reward and
emotional PEs. Predictive neural expressions of reward but not emotional PEs in fronto-insular systems predicted neurofunctional and behavioral punishment decisions. Overall, these findings suggest distinct neurocomputational processes underlie reward and emotional PEs which uniquely impact
social decisions.
Traditional decision-making models conceptualize humans as adaptive learners utilizing the differences between expected and actual rewards (prediction errors, PEs) to maximize outcomes, but rarely consider the influence of violations of emotional expectations (emotional PEs) and how it differs from reward PEs. Here, we conducted a fMRI experiment (n = 43) using a modified Ultimatum Game to examine how reward and emotional PEs affect punishment decisions in terms of rejecting unfair offers. Our results revealed that reward relative to emotional PEs exerted a stronger prediction to punishment decisions. On the neural level, the left dorsomedial prefrontal cortex (dmPFC) was strongly activated during reward receipt whereas the emotions engaged the bilateral anterior insula. Reward and emotional PEs were also encoded differently in brain-wide multivariate patterns, with a more sensitive neural signature observed within fronto-insular circuits for reward PE. We further identified a fronto-insular network encompassing the left anterior cingulate cortex, bilateral insula, left dmPFC and inferior frontal gyrus that encoded punishment decisions. In addition, a stronger fronto-insular pattern expression under reward PE predicted more punishment decisions. These findings underscore that reward and emotional violations interact to shape decisions in complex social interactions, while the underlying neurofunctional PEs computations are distinguishable.
Conventional models of decision-making are predicated upon the notion of rational deliberation. However, empirical evidence has increasingly highlighted the pervasive role of bounded rationality in shaping decisional outcomes. The manifestation of bounded rationality is evident through a spectrum of cognitive biases and heuristics, including but not limited to anchoring, availability, the decoy effect, herd behavior, and the nuanced dynamics of reward and punishment, as well as the implications of weighting and framing effects. This prospective study is dedicated to a comprehensive exploration of such multiple factors together with their impacts to the architecture and functionality of decision-making processes, and their further research potentials as well.
