ArticlePDF Available
Neuroscience and Biobehavioral Reviews 151 (2023) 105233
Available online 15 May 2023
0149-7634/© 2023 Elsevier Ltd. All rights reserved.
Objective models of subjective feelings
Stefano Palminteri
, Romane Cecchi
Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Sant´
e et de la Recherche M´
edicale, Paris, France
epartement d´
Etudes Cognitives, Ecole Normale Sup´
erieure, Universit´
e Paris Sciences et Lettres, Paris, France
Subjective feelings (or emotional states, encompassing both emo-
tions and moods) are complex and pervasive phenomena in the human
experience. They are known to be critical to various aspects of human
life, ranging from learning and decision-making to social interactions
(Lerner et al., 2015; Loewenstein and Lerner, 2003; Tiedens et al., 2004;
Tyng et al., 2017). Furthermore, dysfunctions of emotional states can
have disabling consequences, particularly in the context of psychiatric
disorders (Cl´
ery-Melin et al., 2019; Diekhof et al., 2008; McAllister,
1981; Murphy et al., 2001). Thus, understanding the nature of emotional
states seems essential to advance our understanding of human cognition.
Despite extensive research in this area, the cognitive mechanisms un-
derlying subjective feelings remained largely elusive. To ll this gap,
researchers have increasingly turned to computational modeling as a
promising approach to understanding the underlying mechanisms of
emotional states (Bennett et al., 2022; Jofly and Coricelli, 2013; Rut-
ledge et al., 2014; Vinckier et al., 2018). By using quantitative and
systematic methods, computational models provide a powerful tool for
studying these complex phenomena. In this context, three recently
published reviews have explored the contribution of computational
modeling to our understanding of (subjective) emotional states from
different perspectives.
In a rst review, Kao et al. (2023) explore how computational models
can be built and validated to account for subjective feelings and how
they can be used to gain insight into emotions in psychiatric disorders.
The authors propose that incorporating ratings of subjective feelings
into cognitive paradigms can provide a deeper insight into the formation
of emotional states as well as unexplained deviations in behavior when
these emotional states change. The empirical observation of a two-way
interaction between behavior and emotional states forms the basis of
this argument, where emotional states affect behavior, and behavior can
also inuence emotions and moods. The authors contend that this
approach can provide researchers with a more nuanced understanding
of the underlying mechanisms of psychiatric disorders, ultimately
leading to improved diagnosis and treatment.
Focusing on studies using the same approach, the review by Emanuel
and Eldar (2023) proposes a computational framework, rooted in the
reinforcement learning formalism, that seeks to explain how emotional
states are formed from our interactions with the environment. The au-
thors start by highlighting three key computational variables that play a
crucial role in this process: the value of expected reward, the effec-
tiveness of controllable actions in obtaining a reward (in contrast to
situations where outcomes are independent of actions or where the
agent has no control over their actions), and the probability of future
outcomes that are deemed controllable by the agent. They then propose
a clear mapping between dynamic variations in these internal variables
and specic emotions, arguing that these (computationally dened)
emotions interact and are necessary for an agent to behave adaptively in
the presence of limited information in an uncertain world. By concep-
tualizing emotions as distinct types of computation, this model provides
a more unied and comprehensive understanding of their origins. The
authors also suggest that uctuations in emotional states may help
anticipate changes in the environment and prepare an adaptive response
to those changes. This provides a normative (evolutionary) hint as to
why we experience such states.
Expanding on this line of reasoning, a third study by Pessiglione et al.
(2023) explores the adaptive value of mood reactivity to the environ-
ment. To address this, they propose a rened model of mood that in-
corporates the expected values of all possible actions offered by the
environment, allowing for exibility in mood in response to changing
environmental conditions. In extensive model simulations, they found
* Corresponding author at: Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Sant´
e et de la Recherche M´
edicale, Paris, France.
E-mail address: (S. Palminteri).
Contents lists available at ScienceDirect
Neuroscience and Biobehavioral Reviews
journal homepage:
ResearchGate has not been able to resolve any citations for this publication.
Full-text available
Emotions ubiquitously impact action, learning, and perception, yet their essence and role remain widely debated. Computational accounts of emotion aspire to answer these questions with greater conceptual precision informed by normative principles and neurobiological data. We examine recent progress in this regard and find that emotions may implement three classes of computations, which serve to evaluate states, actions, and uncertain prospects. For each of these, we use the formalism of reinforcement learning to offer a new formulation that better accounts for existing evidence. We then consider how these distinct computations may map onto distinct emotions and moods. Integrating extensive research on the causes and consequences of different emotions suggests a parsimonious one-to-one mapping, according to which emotions are integral to how we evaluate outcomes (pleasure & pain), learn to predict them (happiness & sadness), use them to inform our (frustration & content) and others’ (anger & gratitude) actions, and plan in order to realize (desire & hope) or avoid (fear & anxiety) uncertain outcomes.
Full-text available
A wealth of evidence in perceptual and economic decision-making research suggests that the subjective assessment of one option is influenced by the context. A series of studies provides evidence that the same coding principles apply to situations where decisions are shaped by past outcomes, that is, in reinforcement-learning situations. In bandit tasks, human behavior is explained by models assuming that individuals do not learn the objective value of an outcome, but rather its subjective, context-dependent representation. We argue that, while such outcome context-dependence may be informationally or ecologically optimal, it concomitantly undermines the capacity to generalize value-based knowledge to new contexts — sometimes creating apparent decision paradoxes.
Full-text available
A lack of motivation and anhedonia represent frequent and pervasive symptoms in depression, although with poor specificity. Historically described as a response bias, reward-related impairments in depression may account for the important aspects of the cognitive impairments associated with diagnosis of major depressive disorder. Reward processing is a broad psychological construct that can be parsed into 3 distinct components known as “reinforcement learning” (learning), “reward responsiveness” (liking), and “motivation to obtain a reward” (wanting). Depressed patients respond hyposensitively to reward and maladaptively to punishment: this pattern is related to a dysfunction in the frontostriatal systems modulated by the monoamine systems; seems to be observed in medicated and unmedicated patients with depression and in healthy individuals with high levels of anhedonia; and could be observed in patients with a history of depression, even when in full remission. Considered to be cognitive impairments, reward-related-impairments may also constitute part of an underlying neurobiological vulnerability to major depressive disorder (MDD). For example, the reward-related impairment is state dependent and, more or less, correlated with symptom severity in some studies but has also been proposed as being trait like, with endophenotype characteristics, possibly contributing to the persistence of the disease or treatment resistance. The 3 core aspects of reward processing have specific neurobiological correlates that involve the ventral and dorsal striatum, lateral habenula, ventral tegmental area, orbitofrontal cortex, anterior cingulate cortex, and ventromedial and dorsolateral prefrontal cortex. These structures underline the important role of the dopaminergic mesolimbic pathway, but glutamate and serotonin could also have an important role, at least in some aspects of reward-related impairments.
Full-text available
The influence of mood on choices is a well-established but poorly understood phenomenon. Here, we suggest a three-fold neuro-computational account: (1) the integration of positive and negative events over time induce mood fluctuations, (2) which are underpinned by variations in the baseline activities of critical brain valuation regions, (3) which in turn modulate the relative weights assigned to key dimensions of choice options. We validate this model in healthy participants, using feedback in a quiz task to induce mood fluctuations, and a choice task (accepting vs. declining a motor challenge) to reveal their effects. Using fMRI, we demonstrate the pivotal role of the ventromedial prefrontal cortex and anterior insula, in which baseline activities respectively increase and decrease with theoretical mood level and respectively enhance the weighting of potential gains and losses during decision making. The same mechanisms might explain how decisions are biased in mood disorders at longer timescales.
Full-text available
Emotion has a substantial influence on the cognitive processes in humans, including perception, attention, learning, memory, reasoning, and problem solving. Emotion has a particularly strong influence on attention, especially modulating the selectivity of attention as well as motivating action and behavior. This attentional and executive control is intimately linked to learning processes, as intrinsically limited attentional capacities are better focused on relevant information. Emotion also facilitates encoding and helps retrieval of information efficiently. However, the effects of emotion on learning and memory are not always univalent, as studies have reported that emotion either enhances or impairs learning and long-term memory (LTM) retention, depending on a range of factors. Recent neuroimaging findings have indicated that the amygdala and prefrontal cortex cooperate with the medial temporal lobe in an integrated manner that affords (i) the amygdala modulating memory consolidation; (ii) the prefrontal cortex mediating memory encoding and formation; and (iii) the hippocampus for successful learning and LTM retention. We also review the nested hierarchies of circular emotional control and cognitive regulation (bottom-up and top-down influences) within the brain to achieve optimal integration of emotional and cognitive processing. This review highlights a basic evolutionary approach to emotion to understand the effects of emotion on learning and memory and the functional roles played by various brain regions and their mutual interactions in relation to emotional processing. We also summarize the current state of knowledge on the impact of emotion on memory and map implications for educational settings. In addition to elucidating the memory-enhancing effects of emotion, neuroimaging findings extend our understanding of emotional influences on learning and memory processes; this knowledge may be useful for the design of effective educational curricula to provide a conducive learning environment for both traditional “live” learning in classrooms and “virtual” learning through online-based educational technologies.
This volume is a comprehensive roadmap to the burgeoning area of affective sciences, which now spans several disciplines. The Handbook brings together, for the first time, the various strands of inquiry and latest research in the scientific study of the relationship between the mechanisms of the brain and the psychology of mind. In recent years, scientists have made considerable advances in understanding how brain processes shape emotions and are changed by human emotion. Drawing on a wide range of neuroimaging techniques, neuropsychological assessment, and clinical research, scientists are beginning to understand the biological mechanisms for emotions. As a result, researchers are gaining insight into such compelling questions as: How do people experience life emotionally? Why do people respond so differently to the same experiences? What can the face tell us about internal states? How does emotion in significant social relationships influence health? Are there basic emotions common to all humans? This volume brings together the most eminent scholars in the field to present, in sixty original chapters, the latest research and theories in the field. The book is divided into ten sections: Neuroscience; Autonomic Psychophysiology; Genetics and Development; Expression; Components of Emotion; Personality; Emotion and Social Processes; Adaptation, Culture, and Evolution; Emotion and Psychopathology; and Emotion and Health. This major new volume will be an invaluable resource for researchers that will define affective sciences for the next decade.
A stable and neutral mood (euthymia) is commended by both economic and clinical perspectives, because it enables rational decisions and avoids mental illnesses. Here we suggest, on the contrary, that a flexible mood responsive to life events may be more adaptive for natural selection, because it can help adjust the behavior to fluctuations in the environment. In our model (dubbed MAGNETO), mood represents a global expected value that biases decisions to forage for a particular reward. When flexible, mood is updated every time an action is taken, by aggregating incurred costs and obtained rewards. Model simulations show that, across a large range of parameters, flexible agents outperform cold agents (with stable neutral mood), particularly when rewards and costs are correlated in time, as naturally occurring across seasons. However, with more extreme parameters, simulations generate short manic episodes marked by incessant foraging and lasting depressive episodes marked by total inaction. The MAGNETO model therefore accounts for both the function of mood fluctuations and the emergence of mood disorders.
Research in computational psychiatry is dominated by models of behavior. Subjective experience during behavioral tasks is not well understood, even though it should be relevant to understanding the symptoms of psychiatric disorders. Here, we bridge this gap and review recent progress in computational models for subjective feelings. For example, happiness reflects not how well people are doing, but whether they are doing better than expected. This dependence on recent reward prediction errors is intact in major depression, although depressive symptoms lower happiness during tasks. Uncertainty predicts subjective feelings of stress in volatile environments. Social prediction errors influence feelings of self-worth more in individuals with low self-esteem despite a reduced willingness to change beliefs due to social feedback. Measuring affective state during behavioral tasks provides a tool for understanding psychiatric symptoms that can be dissociable from behavior. When smartphone tasks are collected longitudinally, subjective feelings provide a potential means to bridge the gap between lab-based behavioral tasks and real-life behavior, emotion, and psychiatric symptoms.
Mood is an integrative and diffuse affective state that is thought to exert a pervasive effect on cognition and behavior. At the same time, mood itself is thought to fluctuate slowly as a product of feedback from interactions with the environment. Here we present a new computational theory of the valence of mood-the Integrated Advantage model-that seeks to account for this bidirectional interaction. Adopting theoretical formalisms from reinforcement learning, we propose to conceptualize the valence of mood as a leaky integral of an agent's appraisals of the Advantage of its actions. This model generalizes and extends previous models of mood wherein affective valence was conceptualized as a moving average of reward prediction errors. We give a full theoretical derivation of the Integrated Advantage model and provide a functional explanation of how an integrated-Advantage variable could be deployed adaptively by a biological agent to accelerate learning in complex and/or stochastic environments. Specifically, drawing on stochastic optimization theory, we propose that an agent can utilize our hypothesized form of mood to approximate a momentum-based update to its behavioral policy, thereby facilitating rapid learning of optimal actions. We then show how this model of mood provides a principled and parsimonious explanation for a number of contextual effects on mood from the affective science literature, including expectation- and surprise-related effects, counterfactual effects from information about foregone alternatives, action-typicality effects, and action/inaction asymmetry. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
A variety of behavioral and neural phenomena suggest that organisms evaluate outcomes not on an absolute utility scale, but relative to some dynamic and context-sensitive reference or scale. Sometimes, as in foraging tasks, this results in sensible choices; in other situations, like choosing between options learned in different contexts, irrational choices can result. We argue that what unites and demystifies these various phenomena is that the brain's goal is not assessing utility as an end in itself, but rather comparing different options to choose the better one. In the presence of uncertainty, noise, or costly computation, adjusting options to the context can produce more accurate choices.