Randall C. O’Reilly

• PhD
• Professor (Full) at University of California, Davis

Many tasks require flexibly modifying perception and behavior based on current goals. Humans can retrieve episodic memories from days to years ago, using them to contextualize and generalize behaviors across novel but structurally related situations. The brain's ability to control episodic memories based on task demands is often attributed to inter...

The entorhinal cortex (EC) provides the major input to the hippocampus (HPC). Numerous computational models on the EC propose that its grid cells serve as a spatial metric, supporting path integration and efficient generalization. However, little is known about how these cells could contribute to episodic memory, which emphasizes episode-specific r...

Some neural representations gradually change across multiple timescales. Here we argue that modeling this “drift” could help explain the spacing effect (the long-term benefit of distributed learning), whereby differences between stored and current temporal context activity patterns produce greater error-driven learning. We trained a neurobiological...

While many theories assume that sleep is critical in stabilizing and strengthening memories, our recent behavioral study (Liu & Ranganath, 2021, Psychonomic Bulletin & Review , 28 [6], 2035–2044) suggests that sleep does not simply stabilize memories. Instead, it plays a more complex role, integrating information across two temporally distinct lear...

The Cambridge Handbook of Computational Cognitive Sciences is a comprehensive reference for this rapidly developing and highly interdisciplinary field. Written with both newcomers and experts in mind, it provides an accessible introduction of paradigms, methodologies, approaches, and models, with ample detail and illustrated by examples. It should...

Background and hypothesis: The neuronal mechanisms that underlie deficits in effort cost computation in schizophrenia (SZ) are poorly understood. Given the role of frontostriatal circuits in valence-oriented motivation, we hypothesized that these circuits are either dysfunctional in SZ or do not appropriately predict behavior in SZ when task condi...

Recent work has shown that abstract, non-spatial relationships between entities or task states are organized into representations called cognitive maps. Here we investigated how cognitive control enables flexible top-down selection of goal-relevant information from multidimensional cognitive maps retrieved from memory. We examined the relationship...

Some neural representations change slowly across multiple timescales. Here we argue that modeling this "drift" could help explain the spacing effect (the long-term benefit of distributed learning), whereby differences between stored and current temporal context activity patterns produce greater error-driven learning. We trained a neurobiologically...

While many theories assume that sleep is critical in stabilizing and strengthening memories, our recent behavioral study (Liu & Ranganath, 2021) suggests that sleep does not simply stabilize memories. Instead, it plays a more complex role, integrating information across two temporally distinct learning episodes. In the current study, we simulated t...

The hippocampus plays a critical role in the rapid learning of new episodic memories. Many computational models propose that the hippocampus is an autoassociator that relies on Hebbian learning (i.e., “cells that fire together, wire together”). However, Hebbian learning is computationally suboptimal as it does not learn in a way that is driven towa...

Bayesian interpretations of neural processing require that biological mechanisms represent and operate upon probability distributions in accordance with Bayes' theorem. Many have speculated that synaptic failure constitutes a mechanism of variational, i.e., approximate, Bayesian inference in the brain. Whereas models have previously used synaptic f...

The thalamus is a key structure in the mammalian brain, providing a hub for communication within and across distributed forebrain networks. Research in this area has undergone a revolution in the last decade, with findings that suggest an expanded role for the thalamus in sensory processing, motor control, arousal regulation, and cognition. Moving...

Neural networks struggle in continual learning settings from catastrophic forgetting: when trials are blocked, new learning can overwrite the learning from previous blocks. Humans learn effectively in these settings, in some cases even showing an advantage of blocking, suggesting the brain contains mechanisms to overcome this problem. Here, we buil...

A hallmark of human intelligence is the ability to adapt to new situations by applying learned rules to new content (systematicity) and thereby enabling an open-ended number of inferences and actions (generativity). Here, we propose that the human brain accomplishes these feats through pathways in the parietal cortex that encode the abstract struct...

Neural networks struggle in continual learning settings from catastrophic forgetting: when trials are blocked, new learning can overwrite the learning from previous blocks. Humans learn effectively in these settings, in some cases even showing an advantage of blocking, suggesting the brain contains mechanisms to overcome this problem. Here, we buil...

The Bayesian brain hypothesis postulates that the brain accurately operates on statistical distributions according to Bayes' theorem. The random failure of presynaptic vesicles to release neurotransmitters may allow the brain to sample from posterior distributions of network parameters, interpreted as epistemic uncertainty. It has not been shown pr...

The hippocampus plays a critical role in the rapid learning of new episodic memories. Many computational models propose that the hippocampus is an autoassociator that relies on Hebbian learning (i.e., “cells that fire together, wire together”). However, Hebbian learning is computationally suboptimal as it modifies weights unnecessarily beyond what...

Infants, adults, non-human primates and non-primates all learn patterns implicitly, and they do so across modalities. The biological evidence supports the hypothesis that the mechanism for this learning is general but computationally local. We hypothesize that the mechanism itself is predictive error-driven learning. We build on recent work that ad...

A hallmark of human intelligence is the ability to adapt to new situations, by applying learned rules to new content (systematicity) and thereby enabling an open-ended number of inferences and actions (generativity). Here, we propose that the human brain accomplishes these feats through pathways in the parietal cortex that encode the abstract struc...

The neural mechanisms supporting flexible relational inferences, especially in novel situations, are a major focus of current research. In the complementary learning systems framework, pattern separation in the hippocampus allows rapid learning in novel environments, while slower learning in neocortex accumulates small weight changes to extract sys...

The neural mechanisms supporting flexible relational inferences, especially in novel situations, are a major focus of current research. In the complementary learning systems framework, pattern separation in the hippocampus allows rapid learning in novel environments, while slower learning in neocortex accumulates small weight changes to extract sys...

How do humans learn from raw sensory experience? Throughout life, but most obviously in infancy, we learn without explicit instruction. We propose a detailed biological mechanism for the widely embraced idea that learning is driven by the differences between predictions and actual outcomes (i.e., predictive error-driven learning). Specifically, num...

Compared to our understanding of positive prediction error signals occurring due to unexpected reward outcomes, less is known about the neural circuitry in humans that drives negative prediction errors during omission of expected rewards. While classical learning theories such as Rescorla–Wagner or temporal difference learning suggest that both typ...

We present a theory and neural network model of the neural mechanisms underlying human decision-making. We propose a detailed model of the interaction between brain regions, under a proposer-predictor-actor-critic framework. This theory is based on detailed animal data and theories of action-selection. Those theories are adapted to serial operation...

Episodic memory retrieval is often viewed as a static process that enables one to re-experience past events. However, there is considerable evidence suggesting that memories can be modified, altered, or strengthened through online retrieval and reactivation during sleep. For example, recent research has demonstrated the importance of testing in lea...

Eye movement parameters are consistently investigated, and the distribution of these parameters are well known. Whereas saccade duration has been studied along with saccade amplitude , the distribution of saccade duration has not yet been reported. We aim to investigate the distribution of saccade duration in several eye movement datasets from the...

Motor dysfunction in youth at clinical high risk (CHR) for psychosis is thought to reflect abnormal neurodevelopment within cortical-subcortical motor circuits and may be important for understanding clinical trajectories of CHR individuals. However, to date, our perspective of brain-behavior relationships has been informed solely by cross-sectional...

Extrapolation -- the ability to make inferences that go beyond the scope of one's experiences -- is a hallmark of human intelligence. By contrast, the generalization exhibited by contemporary neural network algorithms is largely limited to interpolation between data points in their training corpora. In this paper, we consider the challenge of learn...

How does the human brain learn new concepts from raw sensory experience, without explicit instruction? We still do not have a widely-accepted answer to this central question. Here, we propose a detailed biological mechanism for the widely-embraced idea that learning is based on the differences between predictions and actual outcomes (i.e., predicti...

We describe a neurobiologically informed computational model of phasic dopamine signaling to account for a wide range of findings, including many considered inconsistent with the simple reward prediction error (RPE) formalism. The central feature of this PVLV framework is a distinction between a primary value (PV) system for anticipating primary re...

Motivation plays a central role in human behavior and cognition but is not well captured by widely used artificial intelligence (AI) and computational modeling frameworks. This Opinion article addresses two central questions regarding the nature of motivation: what are the nature and dynamics of the internal goals that drive our motivational system...

Several experiments, notably one done by Bruner and Potter (1964), have demonstrated delayed object recognition when viewing a blurred image gradually come into focus. Bruner and Potter (1964) suggested that the wrong answer is held until there is an obvious contradiction. Others have hypothesized that “competitive activation” is responsible for de...

We address the distinction between habitual/automatic vs. goal-directed/controlled behavior, from the perspective of a computational model of the frontostriatal loops. The model exhibits a continuum of behavior between these poles, as a function of the interactive dynamics among different functionally-specialized brain areas, operating iteratively...

Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution. However, human learners readily generalize in this way, e.g. by applying known grammatical rules to novel words. Inspired by work in neuroscie...

Computational models of frontal function have made important contributions to understanding how the frontal lobes support a wide range of important functions, in their interactions with other brain areas including, critically, the basal ganglia (BG). We focus here on the specific case of how different frontal areas support goal-directed, motivated...

How does the neocortex learn and develop the foundations of all our high-level cognitive abilities? We present a comprehensive framework spanning biological, computational, and cognitive levels, with a clear theoretical continuity between levels, providing a coherent answer directly supported by extensive data at each level. Learning is based on ma...

One candidate approach to creating artificial general intelligence (AGI) is to imitate the essential computations of human cognition. This process is sometimes called ‘reverse-engineering the brain’ and the end product called ‘neuromorphic.’ We argue that, unlike with other approaches to AGI, anthropomorphic reasoning about behaviour and safety con...

Learning about conditioned inhibitors, which predict omission of outcome delivery, has been relatively understudied compared to learning about reward predictors. Reward omissions lead to inhibition of dopamine neurons, driven by the lateral habenula, an important region that is also involved in learning about predictors of punishment. How could a c...

Visual selection processes in real-world scenes are guided by exogenous and endogenous factors. Endogenous factors correspond to top-down influences like the cognitive tasks or prior knowledge. Object processing is affected by the gist of the scene within which it is embedded and the prior knowledge about the objects. On one hand, it has been shown...

Background: Previous research in patients with anorexia nervosa showed heightened brain response during a taste reward conditioning task and heightened sensitivity to rewarding and punishing stimuli. Here we tested the hypothesis that individuals recovered from anorexia nervosa would also experience greater brain activation during this task as wel...

Significance The subcortical striatum is critical for the planning and execution of motor behavior, and its dysfunction is associated with disorders such as Parkinson’s disease. More recently, the human striatum has also been reported to be involved in heterogeneous nonmotor psychological functions. However, detailed functional mappings of human ps...

Aksentijevic (1) raises issues with our recent report (2) that bear further clarification because they reflect some fundamental confusion about the nature of random sequences. First, it is essential to distinguish statistics for individual elements (i.e., patterns of length one) vs. statistics for patterns consisting of more than one element (i.e.,...

We present a cerebellar architecture with two main characteristics. The first one is that complex spikes respond to increases in sensory errors. The second one is that cerebellar modules associate particular contexts where errors have increased in the past with corrective commands that stop the increase in error. We analyze our architecture formall...

Significance The human mind has a unique capacity to find order in chaos. The way the neocortex integrates information over time enables the mind to capture rich statistical structures embedded in random sequences. We show that a biologically motivated neural network model reacts to not only how often a pattern occurs (mean time) but also when a pa...

The development of intravenous fat emulsion (IVFE) is the culmination of physiological, biochemical, nutritional, and medical scientific advancements. IVFEs have the ability to deliver critical nutritional substrates to the patient. Recent literature purports that they may also play roles in modulation of immune functionality and pulmonary physiolo...

Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, o...

As focus shifts to large-scale network interactions involved in memory, it is becoming increasingly clear that oscillatory dynamics are critically involved. A number of studies have shown a negative correlation between memory retrieval in alpha and beta power, and a positive correlation between retrieval and theta power. In this opinion article, we...

We present a model of the cerebellum with two defining characteristics. The first one is that complex spikes respond to increases in sensory errors. The second one is that cerebellar modules implement forward models that predict how a controller will react to reduce error, instead of predicting the consequence of motor commands. We examine the biol...

We present a comprehensive, novel framework for understanding how the neocortex, including the thalamocortical loops through the deep layers, can support a temporal context representation in the service of predictive learning. Many have argued that predictive learning provides a compelling, powerful source of learning signals to drive the developme...

Standard models of the visual object recognition pathway hold that a largely feedforward process from the retina through inferotemporal cortex leads to object identification. A subsequent feedback process originating in frontoparietal areas through reciprocal connections to striate cortex provides attentional support to salient or behaviorally-rele...

Is human cognition best characterized in terms of the systematic nature of classical symbol processing systems (as argued by Fodor & Pylyshyn, 1988), or in terms of the context-sensitive, embedded knowledge characteristic of classical connectionist or neural network systems? We attempt to bridge these contrasting perspectives in several ways. First...

Current theoretical and computational models of dopamine-based reinforcement learning are largely rooted in the classical behaviorist tradition, and envision the organism as a purely reactive recipient of rewards and punishments, with resulting behavior that essentially reflects the sum of this reinforcement history. This framework is missing some...

Philosophers and cognitive scientists reassess systematicity in the post-connectionist era, offering perspectives from ecological psychology, embodied and distributed cognition, enactivism, and other methodologies. In 1988, Jerry Fodor and Zenon Pylyshyn challenged connectionist theorists to explain the systematicity of cognition. In a highly influ...

The Synthesis of ACT-R and Leabra (SAL) hybrid cognitive architecture is the integration of two theories of cognitive functioning, each itself a highly integrative theory of cognition, ACT-R being predominantly a symbolic production-rule based architecture and Leabra a neural modeling architecture. The combination of the two architectures allows fo...

Cognitive actions are frequently taken with the expectation of a particular result in mind. The expectation is generally based on past experiences, but not necessarily on any specific experience in isolation; rather, the integrated collection of past experiences forms the basis for what we should expect. Although an important part of action executi...

The ability to flexibly, rapidly, and accurately perform novel tasks is a hallmark of human behavior. In our everyday lives we are often faced with arbitrary instructions that we must understand and follow, and we are able to do so with remarkable ease. It has frequently been argued that this ability relies on symbol processing, which depends criti...

Control processes are critical for both facilitating and suppressing memory retrieval, but these processes are not well understood. The current work, inspired by a similar fMRI design [8], used a modified Think/No-Think(TNT) paradigm to investigate the neural signatures of volition over enhancing and suppressing memory retrieval. Previous studies h...

We can learn from the wisdom of others to maximize success. However, it is unclear how humans take advice to flexibly adapt behavior. On the basis of data from neuroanatomy, neurophysiology, and neuroimaging, a biologically plausible model is developed to illustrate the neural mechanisms of learning from instructions. The model consists of two comp...

The learning mechanism in the hippocampus has almost universally been assumed to be Hebbian in nature, where individual neurons in an engram join together with synaptic weight increases to support facilitated recall of memories later. However, it is also widely known that Hebbian learning mechanisms impose significant capacity constraints, and are...

How does the brain learn to recognize objects visually, and perform this difficult feat robustly in the face of many sources of ambiguity and variability? We present a computational model based on the biology of the relevant visual pathways that learns to reliably recognize 100 different object categories in the face of naturally occurring variabil...

We address strategic cognitive sequencing, the "outer loop" of human cognition: how the brain decides what cognitive process to apply at a given moment to solve complex, multistep cognitive tasks. We argue that this topic has been neglected relative to its importance for systematic reasons but that recent work on how individual brain systems accomp...

The Synthesis of ACT-R and Leabra (SAL) hybrid cognitive architecture is the integration of two theories of cognitive functioning, each itself a highly integrative theory of cognition, ACT-R being predominantly a symbolic production-rule based architecture and Leabra a neural modeling architecture. The combination of the two architectures allows fo...

Everyday vision requires robustness to a myriad of environmental factors that degrade stimuli. Foreground clutter can occlude objects of interest, and complex lighting and shadows can decrease the contrast of items. How does the brain recognize visual objects despite these low-quality inputs? On the basis of predictions from a model of object recog...

When we behave according to rules and instructions, our brains interpret abstract representations of what to do and transform them into actual behavior. In order to investigate the neural mechanisms behind this process, we devised an fMRI experiment that explicitly isolated rule interpretation from rule encoding and execution. Our results showed th...

How does the brain bind together visual features that are processed concurrently by different neurons into a unified percept suitable for processes such as object recognition? Here, we describe how simple, commonly accepted principles of neural processing can interact over time to solve the brain’s binding problem. We focus on mechanisms of neural...

Evidence suggests that two regions of the striatum contribute differential support to instrumental response selection. The dorsomedial striatum (DMS) is thought to support expectancy-mediated actions, and the dorsolateral striatum (DLS) is thought to support habits. Currently it is unclear whether these regions store task-relevant information or ju...

Anorexia nervosa (AN) is a severe psychiatric disorder associated with food avoidance and malnutrition. In this study, we wanted to test whether we would find brain reward alterations in AN, compared with individuals with normal or increased body weight. We studied 21 underweight, restricting-type AN (age M 22.5, SD 5.8 years), 19 obese (age M 27.1...

The present experiment tested three hypotheses regarding the function and organization of lateral prefrontal cortex (PFC). The first account (the information cascade hypothesis) suggests that the anterior-posterior organization of lateral PFC is based on the timing with which cue stimuli reduce uncertainty in the action selection process. The secon...

Investigation of sustained and transient activity associated with the information-cascade and levels-of-abstraction hypotheses. (PDF)

Appetitive goal-directed behavior can be associated with a cue-triggered expectancy that it will lead to a particular reward, a process thought to depend on the OFC and basolateral amygdala complex. We developed a biologically informed neural network model of this system to investigate the separable and complementary roles of these areas as the mai...

We present a model of unsupervised learning in the hybrid SAL (Synthesis of ACT-R and Leabra) architecture. This model follows the hypothesis that higher evaluative cognitive mechanisms can serve to provide training signals for perceptual learning. This addresses the problem that supervised learning seems necessary for strong perceptual performance...

This paper reviews the fate of the central ideas behind the complementary learning systems (CLS) framework as originally articulated in McClelland, McNaughton, and O'Reilly (1995). This framework explains why the brain requires two differentially specialized learning and memory systems, and it nicely specifies their central properties (i.e., the hi...

Action selection is the task of doing the right thing at the right time. It requires the assessment of available alternatives, executing those most appropriate, and resolving conflicts among competing goals and possibilities. Using advanced computational modelling, this book explores cutting-edge research into action selection in nature from a wide...

A paradigmatic test of executive control, the n-back task, is known to recruit a widely distributed parietal, frontal, and striatal “executive network,” and is thought to require an equally wide array of executive functions. The mapping of functions onto substrates in such a complex task presents a significant challenge to any theoretical framework...

The ventral visual stream has extensive bidirectional connectivity that has been suggested to promote recurrent processing – what function might this serve in the context of high-level visual processes like object recognition? Previous computational modeling work from our lab has demonstrated that the bidirectional architecture of the ventral visua...

Inhibiting unwanted thoughts, actions and emotions figures centrally in daily life, and the prefrontal cortex (PFC) is widely viewed as a source of this inhibitory control. We argue that the function of the PFC is best understood in terms of representing and actively maintaining abstract information, such as goals, which produces two types of inhib...

A critique of the state of current quantum theory in physics is presented, based on a perspective outside the normal physics training. From this perspective, the acceptance of quantum nonlocality seems unwarranted, and the fundamental assumptions that give rise to it in the first place seem questionable, based on the current status of the quantum t...

Monocular figure-ground segmentation is an important problem in the field of Artificial General Intelligence. A solution to this problem will unlock vast sets of training data, such as Google Images, in which salient objects of interest are situated against complex backgrounds. In order to gain traction on the figure-ground problem we enhanced the...

The neurobiology of bulimia nervosa (BN) is poorly understood. Recent animal literature suggests that binge eating is associated with altered brain dopamine (DA) reward function. In this study, we wanted to investigate DA-related brain reward learning in BN. Ill BN (n = 20, age: mean = 25.2, SD = 5.3 years) and healthy control women (CW) (n = 23, a...

We present a metacognitive classifier implemented within a hybrid architecture that combines the strengths of two existing, mature cognitive architectures: ACT-R and Leabra. The classification of a set of items into previously seen and novel categories (TRAIN and TEST, respectively) is carried out in ACT-R using metacognitive signals supplied by Le...