Geoffrey Schoenbaum

• MD, PhD
• Branch Chief at National Institute on Drug Abuse, National Institutes of Health

Both the orbitofrontal cortex (OFC) and the hippocampus (HC) are implicated in the formation of cognitive maps and their generalization into schemas. However, how these areas interact in supporting this function remains unclear, with some proposals supporting a serial model in which the OFC draws on task representations created by the HC to extract...

Intro The orbitofrontal cortex (OFC) is critical to identifying task structure and to generalizing appropriately across task states with similar underlying or hidden causes 1–6 . This capability is at the heart of OFC’s proposed role in a network responsible for cognitive mapping 7,8 , and its loss can explain many deficits associated with OFC dama...

The inability to suppress actions despite adverse consequences is a hallmark of compulsive behaviors and is amygdala dependent. To study the amygdala's role in responding despite adverse consequences, we compared single-unit activity in the basolateral (BLA) or central (CeA) amygdala before and after reward-seeking under punishment threat. Rats sta...

Schemas allow efficient behavior in new situations, but reliance on them can impair flexibility when new demands conflict, culminating in psychopathology. Evidence implicates the orbitofrontal cortex (OFC) in deploying schemas in new situations congruent with previously acquired knowledge. But how does this role affect learning of a conflicting beh...

Learning when to initiate or withhold actions is essential for survival and requires integration of past experiences with new information to adapt to changing environments. While stable prelimbic cortex (PL) ensembles have been identified during reward learning, it remains unclear how they adapt when contingencies shift. Does the same ensemble adju...

Transient changes in the firing of midbrain dopamine neurons have been closely tied to the unidimensional value-based prediction error contained in temporal difference reinforcement learning models. However, whereas an abundance of work has now shown how well dopamine responses conform to the predictions of this hypothesis, far fewer studies have c...

Learning is driven by prediction errors, which determine what is learned, and salience, which controls the learning rate. Dopamine in the nucleus accumbens correlates with prediction errors, but salience mechanisms are less clear. We hypothesized that acetylcholine acts as a salience signal, as it regulates dopamine-induced plasticity. To test this...

Cue reactivity is the maladaptive neurobiological and behavioral response upon exposure to drug cues and is a major driver of relapse. A widely accepted assumption is that drugs of abuse result in disparate dopamine responses to cues that predict drug vs. natural rewards. The leading hypothesis is that drug-induced dopamine release represents a per...

Adaptive behavior depends on the ability to predict specific events, particularly those related to rewards. Armed with such associative information, we can infer the current value of predicted rewards based on changing circumstances and desires. To support this ability, neural systems must represent both the value and identity of predicted rewards,...

The orbitofrontal cortex (OFC) and hippocampus (HC) both contribute to the cognitive maps that support flexible behavior. Previously, we used the dopamine neurons to measure the functional role of OFC. We recorded midbrain dopamine neurons as rats performed an odor-based choice task, in which expected rewards were manipulated across blocks. We foun...

The orbitofrontal cortex (OFC) is crucial for tracking various aspects of expected outcomes, thereby helping to guide choices and support learning. Our previous study showed that the effects of reward timing and size on the activity of single units in OFC were dissociable when these attributes were manipulated independently ( Roesch et al., 2006)....

Impaired insight in substance use disorder has been argued to reflect a global deficit in using cognitive models to mentally simulate possible future outcomes. The process of mentally simulating outcomes allows us to understand our beliefs about their causes, that is, to have insight and thereby avoid potentially negative outcomes. However, work in...

Cue reactivity is the maladaptive neurobiological and behavioral response upon exposure to drug cues and is a major driver of relapse. The leading hypothesis is that dopamine release by addictive drugs represents a persistently positive reward prediction error that causes runaway enhancement of dopamine responses to drug cues, leading to their path...

Dopamine in the nucleus accumbens ramps up as animals approach desired goals. These ramps have received intense scrutiny because they seem to violate long-held hypotheses on dopamine function. Furthermore, it has been proposed that they are driven by local acetylcholine release, i.e., that they are mechanistically separate from dopamine signals rel...

Chronic psychostimulant use causes long-lasting changes to neural and cognitive function that persist after long periods of abstinence. As cocaine users transition from drug use to abstinence, a parallel transition from hyperactivity to hypoactivity has been found in orbitofrontal-striatal glucose metabolism and striatal D 2 /D 3 -receptor activity...

Outcome-guided behavior requires knowledge about the identity of future rewards. Previous work across species has shown that the dopaminergic midbrain responds to violations in expected reward identity and that the lateral orbitofrontal cortex (OFC) represents reward identity expectations. Here we used network-targeted transcranial magnetic stimula...

Maladaptive decision-making is a hallmark of substance use disorders, though how drugs of abuse alter neural representations supporting adaptive behavior remains poorly understood. Past studies show the orbitofrontal (OFC) and prelimbic (PL) cortices are important for decision making, tracking both task-relevant and latent information. However, pre...

When rats are given discrete choices between social interactions with a peer and opioid or psychostimulant drugs, they choose social interaction, even after extensive drug self-administration experience. Studies show that like drug and nondrug food reinforcers, social interaction is an operant reinforcer and induces dopamine release. However, these...

Chronic psychostimulant use can cause long lasting changes to neural and cognitive function that persist even after long periods of abstinence. As cocaine users transition from drug use to abstinence, a parallel transition from hyperactivity to hypoactivity has been found in orbitofrontal-striatal glucose metabolism, and striatal D2/D3 receptor act...

Dopamine is classically thought to drive learning based on errors in the prediction of rewards and punishments. However, animals also learn to predict cues with no intrinsic value, and it is unclear if such latent learning also relies on dopaminergic prediction errors. Here, we tested this by recording dopamine release in the nucleus accumbens and...

The orbitofrontal cortex (OFC) and hippocampus (HC) are both implicated in forming the cognitive or task maps that support flexible behavior. Previously, we used the dopamine neurons as a sensor or tool to measure the functional effects of OFC lesions (Takahashi et al., 2011). We recorded midbrain dopamine neurons as rats performed an odor-based ch...

Both orbitofrontal cortex (OFC) and hippocampus (HC) are implicated in the formation of cognitive maps and their generalization into schemas. However how these areas interact in supporting this function remains an open question, with some proposals supporting a serial model in which OFC draws upon task representations created by HC to extract key b...

Dopamine neuron activity is tied to the prediction error in temporal difference reinforcement learning models. These models make significant simplifying assumptions, particularly with regard to the structure of the predictions fed into the dopamine neurons, which consist of a single chain of timepoint states. Although this predictive structure can...

There is no single way to represent a task. Indeed, despite experiencing the same task events and contingencies, different subjects may form distinct task representations. As experimenters, we often assume that subjects represent the task as we envision it. However, such a representation cannot be taken for granted, especially in animal experiments...

We use mental models of the world—cognitive maps—to guide behavior. The lateral orbitofrontal cortex (lOFC) is typically thought to support behavior by deploying these maps to simulate outcomes, but recent evidence suggests that it may instead support behavior by underlying map creation. We tested between these two alternatives using outcome-specif...

The discovery that DA transients can be mapped onto the reward prediction errors in temporal difference models is a pinnacle achievement of neuroscience. Yet, there is abundant evidence that DA activity reinforces actions, suggesting it serves as an intrinsically rewarding event. These two possibilities are so conceptually intertwined that it is no...

Recording action potentials extracellularly during behavior has led to fundamental discoveries regarding neural function—hippocampal neurons respond to locations in space,¹ motor cortex neurons encode movement direction,² and dopamine neurons signal reward prediction errors³—observations undergirding current theories of cognition,⁴ movement,⁵ and l...

Studies investigating the neural mechanisms by which associations between cues and predicted outcomes control behavior often use associative learning frameworks to understand the neural control of behavior. These frameworks do not always account for the full range of effects that novelty can have on behavior and future associative learning. Here, i...

Dopamine was first described by George Barger, James Ewens, and Henry Dale in 1910 as an epinephrine-like monoamine compound. Initially believed to be a mere precursor of norepinephrine, it was mostly ignored for the next four decades (Figure 1A). However, in the 1950s Kathleen Montagu showed that dopamine occurred in the brain by itself, and a ser...

There is no single way to represent a task. Indeed, despite experiencing the same task events and contingencies, different subjects may form distinct task representations. As experimenters, we often assume that subjects represent the task as we envision it. However, such a representation cannot be taken for granted, especially in animal experiments...

We use internal models of the external world to guide behavior, but little is known about how these cognitive maps are created . The orbitofrontal cortex (OFC) is typically thought to access these maps to support model-based decision-making, but it has recently been proposed that its critical contribution may be instead to integrate information int...

Of all frontocortical subregions, the anterior cingulate cortex (ACC) has perhaps the most overlapping theories of function.1, 2, 3 Recording studies in rats, humans, and other primates have reported diverse neural responses that support many theories,4, 5, 6, 7, 8, 9, 10, 11, 12 yet nearly all these studies have in common tasks in which one event...

Olavo Amaral and Kleber Neves argue that collaborative projects to confirm other researchers’ experimental results could help to resolve the reproducibility crisis (see Nature 597, 329–331; 2021). In our view as laboratory researchers, it would be more effective in the long term to test the limits of a paper’s conclusions. As the authors point out...

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e. when values have been directly experience...

The orbitofrontal cortex (OFC) has been implicated in goal-directed planning and model-based decision-making. One key prerequisite for model-based decision-making is learning the transition structure of the environment-the probabilities of transitioning from one environmental state to another. In this work, we investigated how the OFC might be invo...

The orbitofrontal cortex (OFC) and hippocampus share striking cognitive and functional similarities. As a result, both structures have been proposed to encode “cognitive maps” that provide useful scaffolds for planning complex behaviors. However, while this function has been exemplified by spatial coding in neurons of hippocampal regions-particular...

Hippocampal place cells represent spatial locations, but it is unclear how they incorporate associations between locations and specific outcomes. A recent study illuminates this issue by showing that place cells in intermediate hippocampus remap their fields following changes in reward.

The orbitofrontal cortex (OFC) has been proposed to encode expected outcomes, which is thought to be important for outcome-directed behavior. However, such neural encoding can also often be explained by the recall of information about the recent past. To dissociate the retrospective and prospective aspects of encoding in the OFC, we designed a nons...

Deep Reinforcement Learning (Deep RL) agents have in recent years emerged as successful models of animal behavior in a variety of complex learning tasks, as exemplified by Song et al. [2017]. As agents are typically trained to mimic an animal subject, the emphasis in past studies on behavior as a means of evaluating the fitness of models to experim...

Animals learn not only what is potentially useful but also what is meaningless and should be disregarded. How this is accomplished is a key but seldom explored question in psychology and neuroscience. Learning to ignore irrelevant cues is evident in latent inhibition-the ubiquitous phenomenon where presenting a cue several times without consequence...

Many decisions are guided by expectations about their outcomes. These expectations can arise from two fundamentally different sources: from direct experience with outcomes and the events and actions that precede them or from mental simulations and inferences when direct experience is missing. Here we discuss four elegant tasks from animal learning...

Theories of orbitofrontal cortex (OFC) function have evolved substantially over the last few decades. There is now a general consensus that the OFC is important for predicting aspects of future events and for using these predictions to guide behavior. Yet the precise content of these predictions and the degree to which OFC contributes to agency con...

This special issue, commissioned after the 4th Quadrennial Meeting on Orbitofrontal Cortex Function held in Paris in November of 2019 (https://ofc2019.sciencesconf.org/), is intended to provide a snapshot of this ongoing transformation; we hope that the ideas presented herein will provide a foundation for the next stage in the evolution of our unde...

Experimental research controls for past experience, yet prior experience influences how we learn. Here, we tested whether we could recruit a neural population that usually encodes rewards to encode aversive events. Specifically, we found that GABAergic neurons in the lateral hypothalamus (LH) were not involved in learning about fear in naïve rats....

How do we learn about what to learn about? Specifically, how do the neural elements in our brain generalize what has been learned in one situation to recognize the common structure of—and speed learning in—other, similar situations? We know this happens because we become better at solving new problems—learning and deploying schemas1–5—through exper...

Animals can categorize the environment into "states," defined by unique sets of available action-outcome contingencies in different contexts. Doing so helps them choose appropriate actions and make accurate outcome predictions when in each given state. State maps have been hypothesized to be held in the orbitofrontal cortex (OFC), an area implicate...

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienc...

Substance use disorders (SUDs) are characterized by maladaptive behavior. The ability to properly adjust behavior according to changes in environmental contingencies necessitates the interlacing of existing memories with updated information. This can be achieved by assigning learning in different contexts to compartmentalized "states." Though not o...

Importance The tools and insights of behavioral neuroscience grow apace, yet their clinical application is lagging. Observations This article suggests that associative learning theory may be the algorithmic bridge to connect a burgeoning understanding of the brain with the challenges to the mind with which all clinicians and researchers are concer...

When direct experience is unavailable, animals and humans can imagine or infer the future to guide decisions. Behavior based on direct experience versus inference may recruit partially distinct brain circuits. In rodents, the orbitofrontal cortex (OFC) contains neural signatures of inferred outcomes, and OFC is necessary for behavior that requires...

Theories of orbitofrontal cortex (OFC) function have evolved substantially over the last few decades. There is now a general consensus that the OFC is important for predicting aspects of future events and for using these predictions to guide behavior. Yet the precise content of these predictions and the degree to which OFC contributes to agency con...

Learning the transition structure of the environment – the probabilities of transitioning from one environmental state to another – is a key prerequisite for goal-directed planning and model-based decision making. To investigate the role of the orbitofrontal cortex (OFC) in goal-directed planning and decision making, we used fMRI to assess univaria...

The orbitofrontal cortex (OFC) has been proposed to encode expected outcomes, which is thought to be important for outcome-directed behavior. However, such neural encoding can also often be explained by the recall of information about the recent past. To dissociate the retrospective and prospective aspects of encoding in the OFC, we designed a non-...

The orbitofrontal cortex (OFC) is proposed to be critical to economic decision making. Yet one can inactivate OFC without affecting well-practiced choices. One possible explanation of this lack of effect is that well-practiced decisions are codified into habits or configural-based policies not normally thought to require OFC. Here, we tested this i...

The orbitofrontal cortex (OFC) is necessary for inferring value in tests of model-based reasoning, including in sensory preconditioning. This involvement could be accounted for by representation of value or by representation of broader associative structure. We recently reported neural correlates of such broader associative structure in OFC during...

The orbitofrontal cortex (OFC) is necessary for inferring value in tests of model-based reasoning, including in sensory preconditioning. This involvement could be accounted for by representation of value or by representation of broader associative structure. We recently reported neural correlates of such broader associative structure in OFC during...

The orbitofrontal cortex (OFC) is necessary for inferring value in tests of model-based reasoning, including in sensory preconditioning. This involvement could be accounted for by representation of value or by representation of broader associative structure. We recently reported neural correlates of such broader associative structure in OFC during...

The orbitofrontal cortex (OFC) is necessary for value inference in tests of model-based reasoning. This ability could be accounted for by either representation of value or by representation of broader associative structure. Our lab recently reported correlates of both value and of valueless associative structure in OFC using single-unit recording (...

Smmary Sensory areas have been shown to be influenced by higher-order cognitive processes. Yet how do these top-down processes affect decisions? A recent study has revealed a dynamic evolution of neural activity from sensory discrimination to choice in rodent taste cortex.

Decisions are typically guided by what we have experienced in the past. However, when direct experience is unavailable, animals and humans can imagine or infer the future to make choices. Outcome expectations that are based on direct experience and inference may compete for guiding behavior [1, 2], and they may recruit distinct but overlapping brai...

Reward-evoked dopamine transients are well established as prediction errors. However, the central tenet of temporal difference accounts-that similar transients evoked by reward predictive cues also function as errors-remains untested. In the present communication we addressed this by showing that optogenetically shunting dopamine activity at the st...

Internal representations of relationships between events in the external world can be utilized to infer outcomes when direct experience is lacking. This process is thought to involve the orbitofrontal cortex (OFC) and hippocampus (HPC), but there is little evidence regarding the relative role of these areas and their interactions in inference. Here...

Dopamine neurons are proposed to signal the reward prediction error in model-free reinforcement learning algorithms. This term represents the unpredicted or ‘excess’ value of the rewarding event, value that is then added to the intrinsic value of any antecedent cues, contexts or events. To support this proposal, proponents cite evidence that artifi...

Outcome-guided behavior requires knowledge about the current value of expected outcomes. Such behavior can be isolated in the reinforcer devaluation task, which assesses the ability to infer the current value of specific rewards after devaluation. Animal lesion studies demonstrate that orbitofrontal cortex (OFC) is necessary for normal behavior in...

Neural correlates implicate the orbitofrontal cortex (OFC) in value-based or economic decision making [1-3]. Yet inactivation of OFC in rats performing a rodent version of the standard economic choice task is without effect [4, 5], a finding more in accord with ideas that the OFC is primarily necessary for behavior when new information must be take...

Dopamine neurons respond to errors in predicting value-neutral sensory information. These data, combined with causal evidence that dopamine transients support sensory-based associative learning, suggest that the dopamine system signals a multidimensional prediction error. Yet such complexity is not evident in individual neuron or average neural act...

Both hippocampus (HPC) and orbitofrontal cortex (OFC) have been shown to be critical for behavioral tasks that require use of an internal model or cognitive map, composed of the states and the relationships between them, which define the current environment or task at hand. One general idea is that the HPC provides the cognitive map, which is then...

Outcome-guided behavior requires knowledge about the current value of expected outcomes. Such behavior can be isolated in the reinforcer devaluation task, which assesses the ability to infer the current value of rewards after devaluation. Animal lesion studies demonstrate that orbitofrontal cortex (OFC) is necessary for normal behavior in this task...

Dopamine neurons respond to errors in predicting value-neutral sensory information. These data, combined with causal evidence that dopamine transients support sensory-based associative learning, suggest that the dopamine system signals a multidimensional prediction error. Yet such complexity is not evident in individual neuron or average neural act...

The firing of dopaminergic midbrain neurons is thought to reflect prediction errors (PE) that depend on the difference between the value of expected and received rewards. However, recent work has demonstrated that unexpected changes in value-neutral outcome features, such as identity, can evoke similar responses. It remains unclear whether the magn...

Dopamine neurons fire transiently in response to unexpected rewards. These neural correlates are proposed to signal the reward prediction error described in model-free reinforcement learning algorithms. This error term represents the unpredicted or excess value of the rewarding event. In model-free reinforcement learning, this value is then stored...

The orbitofrontal cortex (OFC) has long been implicated in signaling information about expected outcomes to facilitate adaptive or flexible behavior. Current proposals focus on signaling of expected value versus the representation of a value-agnostic cognitive map of the task. While often suggested as mutually exclusive, these alternatives may repr...

Reward-evoked dopamine is well-established as a prediction error. However the central tenet of temporal difference accounts - that similar transients evoked by reward-predictive cues also function as errors - remains untested. To address this, we used two phenomena, second-order conditioning and blocking, in order to examine the role of dopamine in...

Addiction is a disorder of behavioral control and learning. While this may reflect pre-existing propensities, drug use also clearly contributes by causing changes in outcome processing in prefrontal and striatal regions. This altered processing is associated with behavioral deficits, including changes in learning. These areas provide critical input...

Making decisions in environments with few choice options is easy. We select the action that results in the most valued outcome. Making decisions in more complex environments, where the same action can produce different outcomes in different conditions, is much harder. In such circumstances, we propose that accurate action selection relies on top-do...

The orbitofrontal cortex (OFC) has long been implicated in signaling information about expected outcomes to facilitate adaptive or flexible behavior. Current proposals focus on signaling of expected reward values versus the representation of a value-agnostic cognitive map of the task. While often suggested as mutually exclusive, these alternatives...

Midbrain dopamine neurons are commonly thought to report a reward prediction error (RPE), as hypothesized by reinforcement learning (RL) theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here, we develop a new theory of dopamine...

How are decisions made between different goods? One theory spanning several fields of neuroscience proposes that their values are distilled to a single common neural currency, the calculation of which allows for rational decisions. The orbitofrontal cortex (OFC) is thought to play a critical role in this process, based on the presence of neural cor...

In the version of this article initially published, the laser activation at the start of cue X in experiment 1 was described in the first paragraph of the Results and in the third paragraph of the Experiment 1 section of the Methods as lasting 2 s; in fact, it lasted only 1 s. The error has been corrected in the HTML and PDF versions of the article...

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change...

Development of sign tracking and DA signals over training averaged across rats. (A-B) Average beam break (solid) and lever press (dashed) rate for 120-s (A) and 60-s (B) ITI groups. (C-D) Average lever press rate for 120-s (C) and 60-s (D) ITI groups. Data are the same as in A and B but with a smaller scale so that differences and timing can be bet...

Underlying data for S3 Fig. (XLSX)

Underlying data for S4 Fig. (XLSX)

Task and electrode placement. (A) DA release was recorded during a standard Pavlovian conditioned approach behavior task for 10 d. Each behavioral session consisted of 25 trials presented at a random time interval of either 60 s (± 30; n = 7 rats) or 120 s (± 30; n = 12 rats). (B-C) Placement of chronic recording electrodes within the NAc core [39]...

Underlying data for Fig 2. (XLSX)