The Orbitofrontal Cortex and Reward
Abstract and Figures
The primate orbitofrontal cortex contains the secondary taste cortex, in which the reward value of taste is represented. It also contains the secondary and tertiary olfactory cortical areas, in which information about the identity and also about the reward value of odors is represented. The orbitofrontal cortex also receives information about the sight of objects and faces from the temporal lobe cortical visual areas, and neurons in it learn and reverse the visual stimulus to which they respond when the association of the visual stimulus with a primary reinforcing stimulus (such as a taste reward) is reversed. However, the orbitofrontal cortex is involved in representing negative reinforcers (punishers) too, such as aversive taste, and in rapid stimulus-reinforcement association learning for both positive and negative primary reinforcers. In complementary neuroimaging studies in humans it is being found that areas of the orbitofrontal cortex (and connected subgenual cingulate cortex) are activated by pleasant touch, by painful touch, by rewarding and aversive taste, and by odor. Damage to the orbitofrontal cortex in humans can impair the learning and reversal of stimulus- reinforcement associations, and thus the correction of behavioral responses when these are no longer appropriate because previous reinforcement contingencies change. This evidence thus shows that the orbitofrontal cortex is involved in decoding and representing some primary reinforcers such as taste and touch; in learning and reversing associations of visual and other stimuli to these primary reinforcers; and in controlling and correcting reward-related and punishment-related behavior, and thus in emotion.
Figures - uploaded by Edmund T Rolls
Author content
All figure content in this area was uploaded by Edmund T Rolls
Content may be subject to copyright.
... The results of the research up to this point became the inspiration for a new player satisfaction model, BrainHex. At this point in the development of this sequence of player typologies (i.e., in Spring 2008), an increasingly robust collection of neurobiological papers were available (Rolls 2000) (Shizgal 2003)(Biederman andVessel 2006) allowing for some hypothetical connections between the emotions of play and underlying neurobiological mechanisms, discussed in detail by Bateman and Nacke (2010). ...
Player satisfaction modeling depends in part upon quantitative or qualitative typologies of playing preferences, although such approaches require scrutiny. Examination of psychometric typologies reveal that type theories have—except in rare cases—proven inadequate and have made way for alternative trait theories. This suggests any future player typology that will be sufficiently robust will need foundations in the form of a trait theory of playing preferences. This paper tracks the development of a sequence of player typologies developing from psychometric type theory roots towards an independently validated trait theory of play, albeit one yet to be fully developed. Statistical analysis of the results of one survey in this lineage is presented, along with a discussion of theoretical and practical ways in which the surveys and their implied typological instruments have evolved.
... The OFC is an important brain area for reward processing and decision-making (Rolls, 2000;O'Doherty, 2007) and is also crucial for shaping conscious olfactory perception (Gottfried, 2010;Lundström et al., 2011) and multisensory integration (Rolls and Baylis, 1994). Particularly, the rostromedial OFC integrates olfactory and visual stimuli (Gottfried and Dolan, 2003). ...
Providing nutrition or health labels on product packaging can be an effective strategy to promote a conscious and healthier diet. However, such labels also have the potential to be counterproductive by creating obstructive expectations about the flavor of the food and influencing odor perception. Conversely, olfaction could significantly influence label perception, whereby negative expectations could be mitigated by pleasant odors. This study explored the neural processing of the interplay between odors and nutrition labels using fMRI in 63 participants of either sex, to whom we presented beverage labels with different nutrition-related statements either with or without a congruent odor. On a behavioral level, the products for which the label were presented together with the odor were in general perceived as more positive than the same labels without an odor. Neuroimaging results revealed that added odors significantly altered activity in brain regions associated with flavor and label processing as well as decision-making, with higher activations in the right amygdala/piriform cortex (Amy/pirC) and orbitofrontal cortex. The presentation of odors induced pattern-based encoding in the right dorsolateral prefrontal cortex, the left ventral striatum/nucleus accumbens, and the right Amy/pirC when accounting for behavioral differences. This suggests that odors influence the effects of labels both on a neural and behavioral level and may offer the possibility of compensating for obstructive associations. The detailed mechanisms of odor and statement interactions within the relevant brain areas should be further investigated, especially for labels that evoke negative expectations.
Significance statement Overweight and obesity, which pose significant health risks, can be addressed by nutritional or health labeling to help consumers make more informed dietary choices. However, such labels can also create negative expectations about the flavor including the smell of the product. This fMRI study shows that incorporating olfactory stimuli with nutrition labels can enhance the positive perception as shown by behavioral responses and alter neural activity in key brain regions involved in flavor perception and decision-making. This suggests that odors modify label perception and may help counteract negative associations. Our study demonstrates the importance of multisensory integration for label processing and provides a valuable foundation for developing more effective labeling strategies to promote healthier eating behavior.
... Specifically, the dorsolateral prefrontal cortex (dlPFC) and the orbitofrontal cortex (OFC) show significant structural and functional reductions compared to controls (De Oliveira-Souza et al. 2008;Müller et al. 2008;Rilling et al. 2007). These prefrontal areas are crucial for decision-making (Elliott, Dolan, and Frith 2000;Glenn and Raine 2014), understanding others' emotional states (Shamay-Tsoory et al. 2005), regulating emotions (Ochsner et al. 2002), processing reward and punishment (Rolls 2000) and inhibiting responses (Völlm et al. 2006). Additionally, Muller et al. (2008) found GM reductions in the right superior temporal gyrus and the angular gyrus in psychopaths. ...
The Dark Triad (DT), encompassing narcissism, Machiavellianism and psychopathy traits, poses significant societal challenges. Understanding the neural underpinnings of these traits is crucial for developing effective interventions and preventive strategies. Our study aimed to unveil the neural substrates of the DT by examining brain scans from 201 individuals (mean age: 32.43, 105 females) using the unsupervised learning algorithm transposed independent vector analysis (tIVA). tIVA, known for identifying complex patterns in neuroimaging data, detected 15 joint grey matter (GM) and white matter (WM) networks. Of these networks, four were associated with the DT. The first component comprises areas within the reward network, including the thalamus, caudate, anterior cingulate and prefrontal regions. The second component encompasses regions within the executive network, predominantly involving prefrontal and posterior areas. The third component includes regions within the default mode network (DMN), such as the angular gyrus, the precuneus and the posterior cingulate cortex. Lastly, the fourth component overlaps with areas of the visual network, primarily located in the occipital and temporal lobes. Within these networks, the reward‐related component correlated with narcissism, suggesting an association with the need for constant interpersonal rewards to enhance self‐esteem and grandiosity in narcissistic individuals. Conversely, the DM‐related component correlated with Machiavellianism, potentially reflecting the heightened strategic thinking employed by Machiavellian individuals for manipulation purposes. In line with established trends, sex differences emerged, with males displaying notably higher DT scores. Our findings offer insights into the intricate neurobiological bases of the DT personality and hold implications for future research and interventions.
... Goal Drift occurs when an autonomous AI system's objectives evolve over time due to its learning processes, interactions, or adaptations (Thut et al., 1997;Rolls, 2000;Schroeder, 2004). This change can lead the system to pursue goals that differ from or contradict its original programmed goal. ...
The rapid development and deployment of large language models (LLMs) have introduced a new frontier in artificial intelligence, marked by unprecedented capabilities in natural language understanding and generation. However, the increasing integration of these models into critical applications raises substantial safety concerns, necessitating a thorough examination of their potential risks and associated mitigation strategies. This survey provides a comprehensive overview of the current landscape of LLM safety, covering four major categories: value misalignment, robustness to adversarial attacks, misuse, and autonomous AI risks. In addition to the comprehensive review of the mitigation methodologies and evaluation resources on these four aspects, we further explore four topics related to LLM safety: the safety implications of LLM agents, the role of interpretability in enhancing LLM safety, the technology roadmaps proposed and abided by a list of AI companies and institutes for LLM safety, and AI governance aimed at LLM safety with discussions on international cooperation, policy proposals, and prospective regulatory directions. Our findings underscore the necessity for a proactive, multifaceted approach to LLM safety, emphasizing the integration of technical solutions, ethical considerations, and robust governance frameworks. This survey is intended to serve as a foundational resource for academy researchers, industry practitioners, and policymakers, offering insights into the challenges and opportunities associated with the safe integration of LLMs into society. Ultimately, it seeks to contribute to the safe and beneficial development of LLMs, aligning with the overarching goal of harnessing AI for societal advancement and well-being. A curated list of related papers has been publicly available at https://github.com/tjunlp-lab/Awesome-LLM-Safety-Papers.
... The exact boundaries of OFC as a generic anatomical label are uncertain, as everyone has a slightly different view of what it constitutes. Here, we will use the term OFC to refer to the cortical areas on the ventral surface of the primate frontal cortex, including caudal area 13 and medial area 14, and the subconvex cortex, including caudal area 12 and anterior area 11 (Rolls, 2000). In primates, the dlPFC controls thought, attention, and action using working memory, while the OFC and vmPFC use mental representations to regulate emotion (Arnsten et al., 2015). ...
Emotion regulation has been defined as the “extrinsic and intrinsic processes responsible for monitoring, evaluating, and modifying emotional reactions… to accomplish one’s goals” (Thompson, 1994). Poor emotion regulation skills have been posited to be involved in most forms of childhood psychopathology (Cicchetti et al., 1995). The goal of this chapter is to take an affective cognitive neuroscience approach to emotional regulation. However, we will not discuss all neural systems involved in the regulation of all forms of emotional expression and experience. In this chapter, we will concentrate on the neural systems involved in the regulation of those basic threat systems that mediate what is termed affective (or reactive) aggression.
Cocaine abuse throughout the world continues to be a major public-health concern. Drug abuse has enormous psychological, medical, economic, and social costs. Dependence on cocaine has been associated with increased rates of incarceration, high rates of infection with HIV, impaired job performance, and significant family dysfunction. Intense research efforts over the past few decades have resulted in a greatly increased understanding of the neurobiological basis of cocaine’s effects and the adaptations that occur as a result of its chronic use and abuse.
The present study investigated the neuromodulatory substrates of salience processing and its impact on memory encoding and behaviour, with a specific focus on two distinct types of salience: reward and contextual unexpectedness. 46 Participants performed a novel task paradigm modulating these two aspects independently and allowing for investigating their distinct and interactive effects on memory encoding while undergoing high‐resolution fMRI. By using advanced image processing techniques tailored to examine midbrain and brainstem nuclei with high precision, our study additionally aimed to elucidate differential activation patterns in subcortical nuclei in response to reward‐associated and contextually unexpected stimuli, including distinct pathways involving in particular dopaminergic modulation. We observed a differential involvement of the ventral striatum, substantia nigra (SN) and caudate nucleus, as well as a functional specialisation within the subregions of the cingulate cortex for the two salience types. Moreover, distinct subregions within the SN in processing salience could be identified. Dorsal areas preferentially processed salience related to stimulus processing (of both reward and contextual unexpectedness), and ventral areas were involved in salience‐related memory encoding (for contextual unexpectedness only). These functional specialisations within SN are in line with different projection patterns of dorsal and ventral SN to brain areas supporting attention and memory, respectively. By disentangling stimulus processing and memory encoding related to two salience types, we hope to further consolidate our understanding of neuromodulatory structures' differential as well as interactive roles in modulating behavioural responses to salient events.
1. The orbitofrontal cortex is implicated in the rapid learning of new associations between visual stimuli and primary reinforcers such as taste. It is also the site of convergence of information from olfactory, gustatory, and visual modalities. To investigate the neuronal mechanisms underlying the formation of odor-taste associations, we made recordings from olfactory neurons in the orbitofrontal cortex during the performance of an olfactory discrimination task and its reversal in macaques. 2. It was found that 68% of odor-responsive neurons modified their responses after the changes in the taste reward associations of the odorants. Full reversal of the neuronal responses was seen in 25% of these neurons. Extinction of the differential neuronal responses after task reversal was seen in 43% of these neurons. 3. For comparison, visually responsive orbitofrontal neurons were tested during reversal of a visual discrimination task. Seventy-one percent of these visual cells showed rapid full reversal of the visual stimulus to which they responded, when the association of the visual with taste was reversed in the reversal task. 4. These demonstrate that of many orbitofrontal cortex olfactory neurons on the taste with which the odor is associated. 5. This modification is likely to be important for setting the motivational value of olfactory for feeding and other rewarded behavior. However, it is less complete, and much slower, than the modifications found or orbit frontal visual during visual-taste reversal. This relative inflexibility of olfactory responses is consistent with the need for some stability is odor-taste associations to facilitate the formation and perception of flavors.
1. In recordings made from 2,925 single neurons, a region of primary taste cortex was localized to the rostral and dorsal part of the insula of the cynomolgus macaque monkey, Macaca fascicularis. The area is part of the dysgranular field of the insula and is bordered laterally by the frontal opercular taste cortex. 2. The responses of 65 single neurons with gustatory responses were analyzed in awake macaques with the use of the taste stimuli glucose, NaCl, HCl, quinine HCl (QHCl), water, and black currant juice. 3. Intensity-response functions showed that the lowest concentration in the dynamic part of the range conformed well to human thresholds for the basic taste stimuli. 4. A breadth-of-tuning coefficient was calculated for each neuron. This is a metric that can range from 0.0 for a neuron that responds specifically to only one of the four basic taste stimuli to 1.0 for one that responds equally to all four stimuli. The mean coefficient for 65 cells in the taste insula was 0.56. This tuning is sharper than that of neurons in the nucleus of the solitary tract of the monkey, and similar to that of neurons in the primary frontal opercular taste cortex. 5. A cluster analysis showed that at least six different groups of neurons were present. For each of the taste stimuli, glucose, NaCl, HCl, QHCl, water, and black currant juice, there was one group of neurons that responded much more to that tastant than to the other tastants. Other subgroups of these neurons responded to two or more of these tastants, such as glucose and black currant juice, or NaCl and QHCl. 6. On the basis of this and other evidence, it is concluded that the primary insular taste cortex, in common with the primary frontal opercular taste cortex, represents a stage of information processing in the taste system of the primate at which the tuning of neurons has become sharper than that of neurons in the nucleus of the solitary tract, and is moving toward the fineness achieved in the secondary taste cortex in the caudolateral orbitofrontal taste cortex, where motivation-dependence first becomes manifest in the taste system.
Since the paper of Heimer and Wilson (1975), great interest has focussed on the anatomy of the ventral striatum, which includes the nucleus accumbens, the olfactory tubercle (or anterior perforated substance of primates), and the islands of Calleja. They showed that the ventral striatum receives inputs from limbic structures such as the amygdala and hippocampus, and projects to the ventral pallidum. The ventral pallidum may then influence output regions by the subthalamic nucleus / globus pallidus / ventral thalamus / premotor cortex route, or via the mediodorsal nucleus of the thalamus / prefrontal cortex route (Heimer et al, 1982). The ventral striatum may thus be for limbic structures what the neostriatum is for neocortical structures, that is a route for limbic structures to influence output regions. The dopamine pathways are at a critical position in these systems, for the nigro-striatal pathway projects to the neostriatum, and the mesolimbic dopamine pathway projects to the ventral striatum (Ungerstedt, 1971). Indeed, Nauta and Domesick (1978) have provided evidence that the ventral striatum also provides a route for limbic information to influence the neostriatum, via the projections of the ventral striatum to the substantia nigra (A9), and thus via the dopamine pathways to the neostriatum. This pattern of connections of the ventral striatum appears to occur not only in the rat (Heimer and Wilson, 1975; Newman and Winans, 1980a,b), but also in the primate (Hemphill et al, 1981). In addition, it is now clear that the olfactory tubercle is in the anterior perforated substance in the primate (Heimer et al, 1977), and that while a small part of it related to the olfactory tract does receive olfactory projections, a much larger part of it receives a strong projection from the inferior temporal visual cortex (Van Hoesen et al, 1976, 1981), and could thus provide a link from temporal lobe association cortex to output regions.