ArticlePDF Available

Objective models of subjective feelings

May 2023
Neuroscience & Biobehavioral Reviews 151:105233

DOI:10.1016/j.neubiorev.2023.105233

Authors:

Stefano Palminteri

Ecole Normale Supérieure de Paris

Romane Cecchi

Ecole Normale Supérieure de Paris

Content uploaded by Romane Cecchi

Content may be subject to copyright.

Neuroscience and Biobehavioral Reviews 151 (2023) 105233

Available online 15 May 2023

Commentary

Objective models of subjective feelings

Stefano Palminteri

, Romane Cecchi

Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Sant´

e et de la Recherche M´

edicale, Paris, France

D´

epartement d′´

Etudes Cognitives, Ecole Normale Sup´

erieure, Universit´

e Paris Sciences et Lettres, Paris, France

ARTICLE INFO

Keywords

Emotion

Computation

Learning

Subjective feelings (or emotional states, encompassing both emo-

tions and moods) are complex and pervasive phenomena in the human

experience. They are known to be critical to various aspects of human

life, ranging from learning and decision-making to social interactions

(Lerner et al., 2015; Loewenstein and Lerner, 2003; Tiedens et al., 2004;

Tyng et al., 2017). Furthermore, dysfunctions of emotional states can

have disabling consequences, particularly in the context of psychiatric

disorders (Cl´

ery-Melin et al., 2019; Diekhof et al., 2008; McAllister,

1981; Murphy et al., 2001). Thus, understanding the nature of emotional

states seems essential to advance our understanding of human cognition.

Despite extensive research in this area, the cognitive mechanisms un-

derlying subjective feelings remained largely elusive. To ll this gap,

researchers have increasingly turned to computational modeling as a

promising approach to understanding the underlying mechanisms of

emotional states (Bennett et al., 2022; Jofly and Coricelli, 2013; Rut-

ledge et al., 2014; Vinckier et al., 2018). By using quantitative and

systematic methods, computational models provide a powerful tool for

studying these complex phenomena. In this context, three recently

published reviews have explored the contribution of computational

modeling to our understanding of (subjective) emotional states from

different perspectives.

In a rst review, Kao et al. (2023) explore how computational models

can be built and validated to account for subjective feelings and how

they can be used to gain insight into emotions in psychiatric disorders.

The authors propose that incorporating ratings of subjective feelings

into cognitive paradigms can provide a deeper insight into the formation

of emotional states as well as unexplained deviations in behavior when

these emotional states change. The empirical observation of a two-way

interaction between behavior and emotional states forms the basis of

this argument, where emotional states affect behavior, and behavior can

also inuence emotions and moods. The authors contend that this

approach can provide researchers with a more nuanced understanding

of the underlying mechanisms of psychiatric disorders, ultimately

leading to improved diagnosis and treatment.

Focusing on studies using the same approach, the review by Emanuel

and Eldar (2023) proposes a computational framework, rooted in the

reinforcement learning formalism, that seeks to explain how emotional

states are formed from our interactions with the environment. The au-

thors start by highlighting three key computational variables that play a

crucial role in this process: the value of expected reward, the effec-

tiveness of controllable actions in obtaining a reward (in contrast to

situations where outcomes are independent of actions or where the

agent has no control over their actions), and the probability of future

outcomes that are deemed controllable by the agent. They then propose

a clear mapping between dynamic variations in these internal variables

and specic emotions, arguing that these (computationally dened)

emotions interact and are necessary for an agent to behave adaptively in

the presence of limited information in an uncertain world. By concep-

tualizing emotions as distinct types of computation, this model provides

a more unied and comprehensive understanding of their origins. The

authors also suggest that uctuations in emotional states may help

anticipate changes in the environment and prepare an adaptive response

to those changes. This provides a normative (evolutionary) hint as to

why we experience such states.

Expanding on this line of reasoning, a third study by Pessiglione et al.

(2023) explores the adaptive value of mood reactivity to the environ-

ment. To address this, they propose a rened model of mood that in-

corporates the expected values of all possible actions offered by the

environment, allowing for exibility in mood in response to changing

environmental conditions. In extensive model simulations, they found

* Corresponding author at: Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Sant´

e et de la Recherche M´

edicale, Paris, France.

E-mail address: stefano.palminteri@ens.fr (S. Palminteri).

Contents lists available at ScienceDirect

Neuroscience and Biobehavioral Reviews

journal homepage: www.elsevier.com/locate/neubiorev

https://doi.org/10.1016/j.neubiorev.2023.105233

Decoding the Language of Fear: Unveiling Objective and Subjective Indicators in Rodent Models through a Systematic Review and Meta-Analysis

Article

Jan 2024
NEUROSCI BIOBEHAV R

While rodent models are vital for studying mental disorders, the underestimation of construct validity of fear indicators has led to limitations in translating to effective clinical treatments. Addressing this gap, we systematically reviewed 5054 articles from the 1960 s, understanding underlying theoretical advancement, and selected 68 articles with at least two fear indicators for a three-level meta-analysis. We hypothesized correlations between different indicators would elucidate similar functions, while magnitude differences could reveal distinct neural or behavioral mechanisms. Our findings reveal a shift towards using freezing behavior as the primary fear indicator in rodent models, and strong, moderate, and weak correlations between freezing and conditioned suppression ratios, 22-kHz ultrasonic vocalizations, and autonomic nervous system responses, respectively. Using freezing as a reference, moderator analysis shows treatment types and fear stages significantly influenced differences in magnitudes between two indicators. Our analysis supports a two-system model of fear in rodents, where objective and subjective fears could operate on a threshold-based mechanism.

Emotions as Computations

Article

Full-text available

Nov 2022
NEUROSCI BIOBEHAV R

Emotions ubiquitously impact action, learning, and perception, yet their essence and role remain widely debated. Computational accounts of emotion aspire to answer these questions with greater conceptual precision informed by normative principles and neurobiological data. We examine recent progress in this regard and find that emotions may implement three classes of computations, which serve to evaluate states, actions, and uncertain prospects. For each of these, we use the formalism of reinforcement learning to offer a new formulation that better accounts for existing evidence. We then consider how these distinct computations may map onto distinct emotions and moods. Integrating extensive research on the causes and consequences of different emotions suggests a parsimonious one-to-one mapping, according to which emotions are integral to how we evaluate outcomes (pleasure & pain), learn to predict them (happiness & sadness), use them to inform our (frustration & content) and others’ (anger & gratitude) actions, and plan in order to realize (desire & hope) or avoid (fear & anxiety) uncertain outcomes.

A Model of Mood as Integrated Advantage

Article

Full-text available

Sep 2021

Mood is an integrative and diffuse affective state that is thought to exert a pervasive effect on cognition and behavior. At the same time, mood itself is thought to fluctuate slowly as a product of feedback from interactions with the environment. Here we present a new computational theory of the valence of mood-the Integrated Advantage model-that seeks to account for this bidirectional interaction. Adopting theoretical formalisms from reinforcement learning, we propose to conceptualize the valence of mood as a leaky integral of an agent's appraisals of the Advantage of its actions. This model generalizes and extends previous models of mood wherein affective valence was conceptualized as a moving average of reward prediction errors. We give a full theoretical derivation of the Integrated Advantage model and provide a functional explanation of how an integrated-Advantage variable could be deployed adaptively by a biological agent to accelerate learning in complex and/or stochastic environments. Specifically, drawing on stochastic optimization theory, we propose that an agent can utilize our hypothesized form of mood to approximate a momentum-based update to its behavioral policy, thereby facilitating rapid learning of optimal actions. We then show how this model of mood provides a principled and parsimonious explanation for a number of contextual effects on mood from the affective science literature, including expectation- and surprise-related effects, counterfactual effects from information about foregone alternatives, action-typicality effects, and action/inaction asymmetry. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Context-dependent outcome encoding in human reinforcement learning

Article

Full-text available

Jul 2021

A wealth of evidence in perceptual and economic decision-making research suggests that the subjective assessment of one option is influenced by the context. A series of studies provides evidence that the same coding principles apply to situations where decisions are shaped by past outcomes, that is, in reinforcement-learning situations. In bandit tasks, human behavior is explained by models assuming that individuals do not learn the objective value of an outcome, but rather its subjective, context-dependent representation. We argue that, while such outcome context-dependence may be informationally or ecologically optimal, it concomitantly undermines the capacity to generalize value-based knowledge to new contexts — sometimes creating apparent decision paradoxes.

Reward systems and cognitions in Major Depressive Disorder

Article

Full-text available

Nov 2018

A lack of motivation and anhedonia represent frequent and pervasive symptoms in depression, although with poor specificity. Historically described as a response bias, reward-related impairments in depression may account for the important aspects of the cognitive impairments associated with diagnosis of major depressive disorder. Reward processing is a broad psychological construct that can be parsed into 3 distinct components known as “reinforcement learning” (learning), “reward responsiveness” (liking), and “motivation to obtain a reward” (wanting). Depressed patients respond hyposensitively to reward and maladaptively to punishment: this pattern is related to a dysfunction in the frontostriatal systems modulated by the monoamine systems; seems to be observed in medicated and unmedicated patients with depression and in healthy individuals with high levels of anhedonia; and could be observed in patients with a history of depression, even when in full remission. Considered to be cognitive impairments, reward-related-impairments may also constitute part of an underlying neurobiological vulnerability to major depressive disorder (MDD). For example, the reward-related impairment is state dependent and, more or less, correlated with symptom severity in some studies but has also been proposed as being trait like, with endophenotype characteristics, possibly contributing to the persistence of the disease or treatment resistance. The 3 core aspects of reward processing have specific neurobiological correlates that involve the ventral and dorsal striatum, lateral habenula, ventral tegmental area, orbitofrontal cortex, anterior cingulate cortex, and ventromedial and dorsolateral prefrontal cortex. These structures underline the important role of the dopaminergic mesolimbic pathway, but glutamate and serotonin could also have an important role, at least in some aspects of reward-related impairments.

Neuro-computational account of how mood fluctuations arise and affect decision making

Article

Full-text available

Apr 2018

The influence of mood on choices is a well-established but poorly understood phenomenon. Here, we suggest a three-fold neuro-computational account: (1) the integration of positive and negative events over time induce mood fluctuations, (2) which are underpinned by variations in the baseline activities of critical brain valuation regions, (3) which in turn modulate the relative weights assigned to key dimensions of choice options. We validate this model in healthy participants, using feedback in a quiz task to induce mood fluctuations, and a choice task (accepting vs. declining a motor challenge) to reveal their effects. Using fMRI, we demonstrate the pivotal role of the ventromedial prefrontal cortex and anterior insula, in which baseline activities respectively increase and decrease with theoretical mood level and respectively enhance the weighting of potential gains and losses during decision making. The same mechanisms might explain how decisions are biased in mood disorders at longer timescales.

The Influences of Emotion on Learning and Memory

Article

Full-text available

Aug 2017

Emotion has a substantial influence on the cognitive processes in humans, including perception, attention, learning, memory, reasoning, and problem solving. Emotion has a particularly strong influence on attention, especially modulating the selectivity of attention as well as motivating action and behavior. This attentional and executive control is intimately linked to learning processes, as intrinsically limited attentional capacities are better focused on relevant information. Emotion also facilitates encoding and helps retrieval of information efficiently. However, the effects of emotion on learning and memory are not always univalent, as studies have reported that emotion either enhances or impairs learning and long-term memory (LTM) retention, depending on a range of factors. Recent neuroimaging findings have indicated that the amygdala and prefrontal cortex cooperate with the medial temporal lobe in an integrated manner that affords (i) the amygdala modulating memory consolidation; (ii) the prefrontal cortex mediating memory encoding and formation; and (iii) the hippocampus for successful learning and LTM retention. We also review the nested hierarchies of circular emotional control and cognitive regulation (bottom-up and top-down influences) within the brain to achieve optimal integration of emotional and cognitive processing. This review highlights a basic evolutionary approach to emotion to understand the effects of emotion on learning and memory and the functional roles played by various brain regions and their mutual interactions in relation to emotional processing. We also summarize the current state of knowledge on the impact of emotion on memory and map implications for educational settings. In addition to elucidating the memory-enhancing effects of emotion, neuroimaging findings extend our understanding of emotional influences on learning and memory processes; this knowledge may be useful for the design of effective educational curricula to provide a conducive learning environment for both traditional “live” learning in classrooms and “virtual” learning through online-based educational technologies.

The Role of Affect in Decision Making

Chapter

Dec 2002

This volume is a comprehensive roadmap to the burgeoning area of affective sciences, which now spans several disciplines. The Handbook brings together, for the first time, the various strands of inquiry and latest research in the scientific study of the relationship between the mechanisms of the brain and the psychology of mind. In recent years, scientists have made considerable advances in understanding how brain processes shape emotions and are changed by human emotion. Drawing on a wide range of neuroimaging techniques, neuropsychological assessment, and clinical research, scientists are beginning to understand the biological mechanisms for emotions. As a result, researchers are gaining insight into such compelling questions as: How do people experience life emotionally? Why do people respond so differently to the same experiences? What can the face tell us about internal states? How does emotion in significant social relationships influence health? Are there basic emotions common to all humans? This volume brings together the most eminent scholars in the field to present, in sixty original chapters, the latest research and theories in the field. The book is divided into ten sections: Neuroscience; Autonomic Psychophysiology; Genetics and Development; Expression; Components of Emotion; Personality; Emotion and Social Processes; Adaptation, Culture, and Evolution; Emotion and Psychopathology; and Emotion and Health. This major new volume will be an invaluable resource for researchers that will define affective sciences for the next decade.

Origins and consequences of mood flexibility: a computational perspective

Article

Feb 2023

A stable and neutral mood (euthymia) is commended by both economic and clinical perspectives, because it enables rational decisions and avoids mental illnesses. Here we suggest, on the contrary, that a flexible mood responsive to life events may be more adaptive for natural selection, because it can help adjust the behavior to fluctuations in the environment. In our model (dubbed MAGNETO), mood represents a global expected value that biases decisions to forage for a particular reward. When flexible, mood is updated every time an action is taken, by aggregating incurred costs and obtained rewards. Model simulations show that, across a large range of parameters, flexible agents outperform cold agents (with stable neutral mood), particularly when rewards and costs are correlated in time, as naturally occurring across seasons. However, with more extreme parameters, simulations generate short manic episodes marked by incessant foraging and lasting depressive episodes marked by total inaction. The MAGNETO model therefore accounts for both the function of mood fluctuations and the emergence of mood disorders.

Computational models of subjective feelings in psychiatry

Article

Dec 2022
NEUROSCI BIOBEHAV R

Research in computational psychiatry is dominated by models of behavior. Subjective experience during behavioral tasks is not well understood, even though it should be relevant to understanding the symptoms of psychiatric disorders. Here, we bridge this gap and review recent progress in computational models for subjective feelings. For example, happiness reflects not how well people are doing, but whether they are doing better than expected. This dependence on recent reward prediction errors is intact in major depression, although depressive symptoms lower happiness during tasks. Uncertainty predicts subjective feelings of stress in volatile environments. Social prediction errors influence feelings of self-worth more in individuals with low self-esteem despite a reduced willingness to change beliefs due to social feedback. Measuring affective state during behavioral tasks provides a tool for understanding psychiatric symptoms that can be dissociable from behavior. When smartphone tasks are collected longitudinally, subjective feelings provide a potential means to bridge the gap between lab-based behavioral tasks and real-life behavior, emotion, and psychiatric symptoms.

Context-sensitive valuation and learning

Article

Oct 2021

A variety of behavioral and neural phenomena suggest that organisms evaluate outcomes not on an absolute utility scale, but relative to some dynamic and context-sensitive reference or scale. Sometimes, as in foraging tasks, this results in sensible choices; in other situations, like choosing between options learned in different contexts, irrational choices can result. We argue that what unites and demystifies these various phenomena is that the brain's goal is not assessing utility as an end in itself, but rather comparing different options to choose the better one. In the presence of uncertainty, noise, or costly computation, adjusting options to the context can produce more accurate choices.