Fabian Grabenhorst’s research while affiliated with University of Oxford and other places

The successful pursuit of future rewards requires forming an internal goal, followed by planning, decision-making, and progress-tracking over multiple steps. The initial step—forming goals and the plans for obtaining them—involves the subjective valuation of an anticipated reward, considering both the reward’s properties and associated delay and physical-effort costs. Recent findings indicate individuals similarly evaluate cognitive effort over time (Johnson and Most, 2023). Success and failure in these processes have been linked to differential life outcomes and psychiatric conditions. Here we review evidence from single-neuron recordings and neuroimaging studies that implicate the amygdala—a brain structure long associated with cue-reactivity and emotion—in decision-making and the planned pursuit of future rewards (Grabenhorst et al., 2012, 2016, 2019, 2023;Hernadi et al., 2015;Zangemeister et al., 2016). The main findings are that, in behavioral tasks in which future rewards can be pursued through planning and stepwise decision-making, amygdala neurons prospectively encode the value of anticipated rewards and related behavioral plans. Moreover, amygdala neurons predict the stepwise choices to pursue these rewards, signal progress toward goals, and distinguish internally generated (i.e., self-determined) choices from externally imposed actions. Importantly, amygdala neurons integrate the subjective value of a future reward with delay and effort costs inherent in pursuing it. This neural evidence identifies three key computations of the primate amygdala that underlie the pursuit of future rewards: (1) forming a self-determined internal goal based on subjective reward-cost valuations, (2) defining a behavioral plan for obtaining the goal, (3) executing this plan through stepwise decision-making and progress-tracking. Based on this framework, we suggest that amygdala neurons constitute vulnerabilities for dysfunction that contribute to maladaptive reward pursuit in psychiatric and behavioral conditions. Consequently, amygdala neurons may also represent potential targets for behavioral-change interventions that aim to improve individual decision-making.

Risk is a fundamental factor affecting individual and social economic decisions, but its neural correlates are largely unexplored in the social domain. The amygdala, together with the dorsal anterior cingulate cortex (dACC), is thought to play a central role in risk-taking. Here, we investigated in human volunteers ( n = 20; 11 females) how risk (defined as the variance of reward probability distributions) in a social situation affects decisions and concomitant neural activity as measured with fMRI. We found separate variance-risk signals for social and nonsocial outcomes in the amygdala. Specifically, amygdala activity increased parametrically with social reward variance of presented choice options and on separate trials with nonsocial reward variance. Behaviorally, 75% of participants were averse to social risk as estimated in a Becker–DeGroot–Marschak auction-like procedure. The stronger this aversion, the more negative the coupling between risk-related amygdala regions and dACC. This negative relation was significant for social risk attitude but not for the attitude toward variance-risk in juice outcomes. Our results indicate that the amygdala and its coupling with dACC process objective and subjectively evaluated social risk. Moreover, while social risk can be captured with a framework originally established by finance theory for nonsocial risk, the amygdala appears to process social risk largely separately from nonsocial risk.

Risk is a fundamental factor affecting individual and social economic decisions, but its neural correlates are largely unexplored in the social domain. The amygdala, together with the dorsal anterior cingulate cortex (dACC), is thought to play a central role in risk taking. Here, we investigated in human volunteers (n=20; 11 females) how risk (defined as variance of reward probability distributions) in a social situation affects decisions and concomitant neural activity as measured with fMRI. We found social variance-risk signals in the amygdala. Activity in lateral parts of the amygdala increased parametrically with social reward variance of the presented options. Behaviorally, 75% of participants were averse to social risk as estimated in a Becker-DeGroot-Marschak auction-like procedure. The stronger this aversion, the more negative was the coupling between risk-related amygdala regions and dACC. This negative relation was significant for social risk attitude but not for the attitude towards variance-risk in juice outcomes. Our results indicate that the amygdala and its coupling with dACC process objective and subjectively evaluated social risk. Moreover, while social risk can be captured with a framework originally established by finance theory for individual risk, the amygdala appears to processes social risk largely separately from individual risk.

The value of visual stimuli guides learning, decision-making and motivation. Although stimulus values often depend on multiple attributes, how neurons extract and integrate distinct value components from separate cues remains unclear. Here we recorded the activity of amygdala neurons while monkeys viewed sequential cues indicating the probability and magnitude of expected rewards. Amygdala neurons frequently signalled reward probability in an abstract, stimulus-independent code that generalized across cue formats. While some probability-coding neurons were insensitive to magnitude, signalling pure probability rather than value, many neurons showed biphasic responses that signalled probability and magnitude in a dynamic (temporally-patterned) and flexible (reversible) value code. Specific neurons integrated these reward attributes into risk signals that quantified the uncertainty of expected rewards, distinct from value. Population codes were accurate, mutually transferable between value components and expressed differently across amygdala nuclei. Our findings identify amygdala neurons as a substrate for the sequential integration of multiple reward attributes into value and risk.

Although overconsumption of high-fat foods is a major driver of weight gain, the neural mechanisms that link the oral sensory properties of dietary fat to reward valuation and eating behavior remain unclear. Here we combine novel food-engineering approaches with functional neuroimaging to show that the human orbitofrontal cortex (OFC) translates oral sensations evoked by high-fat foods into subjective economic valuations that guide eating behavior. Male and female volunteers sampled and evaluated nutrient-controlled liquid foods that varied in fat and sugar (‘milkshakes’). During oral food processing, OFC activity encoded a specific oral-sensory parameter that mediated the influence of the foods’ fat content on reward value: the coefficient of sliding friction. Specifically, OFC responses to foods in the mouth reflected the smooth, oily texture (i.e., mouthfeel) produced by fatty liquids on oral surfaces. Distinct activity patterns in OFC encoded the economic values associated with particular foods, which reflected the subjective integration of sliding friction with other food properties (sugar, fat, viscosity). Critically, neural sensitivity of OFC to oral texture predicted individuals’ fat preferences in a naturalistic eating test: individuals whose OFC was more sensitive to fat-related oral texture consumed more fat during ad libitum eating. Our findings suggest that reward systems of the human brain sense dietary fat from oral sliding friction, a mechanical food parameter that likely governs our daily eating experiences by mediating interactions between foods and oral surfaces. These findings identify a specific role for the human OFC in evaluating oral food textures to mediate preference for high-fat foods. Significance Statement Fat and sugar enhance the reward value of food by imparting a sweet taste and rich mouthfeel but also contribute to overeating and obesity. Here we used a novel food-engineering approach to realistically quantify the physical-mechanical properties of high-fat liquid foods on oral surfaces and used functional neuroimaging while volunteers sampled these foods and placed monetary bids to consume them. We found that a specific area of the brain’s reward system—the orbitofrontal cortex—detects the smooth texture of fatty foods in the mouth and links these sensory inputs to economic valuations that guide eating behavior. These findings can inform the design of low-calorie fat-replacement foods that mimic the impact of dietary fat on oral surfaces and neural reward systems.

Primates make decisions visually by shifting their view from one object to the next, comparing values between objects, and choosing the best reward, even before acting. Here, we show that when monkeys make value-guided choices, amygdala neurons encode their decisions in an abstract, purely internal representation defined by the monkey’s current view but not by specific object or reward properties. Across amygdala subdivisions, recorded activity patterns evolved gradually from an object-specific value code to a transient, object-independent code in which currently viewed and last-viewed objects competed to reflect the emerging view-based choice. Using neural-network modeling, we identified a sequence of computations by which amygdala neurons implemented view-based decision making and eventually recovered the chosen object’s identity when the monkeys acted on their choice. These findings reveal a neural mechanism in the amygdala that derives object choices from abstract, view-based computations, suggesting an efficient solution for decision problems with many objects.

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of metrics based on information theory in examining adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.

In Reinforcement Learning (RL), animals choose by assigning values to options and learn by updating these values from reward outcomes. This framework has been instrumental in identifying fundamental learning variables and their neuronal implementations. However, canonical RL models do not explain how reward values are constructed from biologically critical intrinsic reward components, such as nutrients. From an ecological perspective, animals should adapt their foraging choices in dynamic environments to acquire nutrients that are essential for survival. Here, to advance the biological and ecological validity of RL models, we investigated how (male) monkeys adapt their choices to obtain preferred nutrient rewards under varying reward probabilities. We found that the rewards’ nutrient composition strongly influenced learning and choices. The animals’ preferences for specific nutrients (sugar, fat) affected how they adapted to changing reward probabilities: the history of recent rewards influenced monkeys’ choices more strongly if these rewards contained the monkey’s preferred nutrients (‘nutrient-specific reward history’). The monkeys also chose preferred nutrients even when they were associated with lower reward probability. A nutrient-sensitive RL model captured these processes: it updated the values of individual sugar and fat components of expected rewards based on experience and integrated them into subjective values that explained the monkeys’ choices. Nutrient-specific reward prediction errors guided this value-updating process. Our results identify nutrients as important reward components that guide learning and choice by influencing the subjective value of choice options. Extending RL models with nutrient-value functions may enhance their biological validity and uncover nutrient-specific learning and decision variables. SIGNIFICANCE STATEMENT: Reinforcement learning (RL) is an influential framework that formalizes how animals learn from experienced rewards. Although 'reward’ is a foundational concept in RL theory, canonical RL models cannot explain how learning depends on specific reward properties, such as nutrients. Intuitively, learning should be sensitive to the reward’s nutrient components, to benefit health and survival. Here we show that the nutrient (fat, sugar) composition of rewards affects monkeys’ choices and learning in an RL paradigm, and that key learning variables including ‘reward history’ and ‘reward prediction error’ should be modified with nutrient-specific components to account for monkeys’ behavior in our task. By incorporating biologically critical nutrient rewards into the RL framework our findings help advance the ecological validity of RL models.

Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied a set of metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of entropy-based metrics in studying adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.