Randall C. O’Reilly

Randall C. O’Reilly
University of California, Davis | UCD · Psychology and Computer Science and Center for Neuroscience

PhD

About

192
Publications
40,487
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
21,921
Citations
Introduction
Computational cognitive neuroscience, developing biologically-based large-scale system's neuroscience models of cognitive phenomena. Focused on predictive learning in the thalamocortical loops, motivated cognitive control in prefrontal cortex and basal ganglia, and flexible structured knowledge manipulation in parietal and hippocampal networks.
Additional affiliations
August 1997 - present
University of Colorado Boulder
Position
  • Professor (Full)

Publications

Publications (192)
Chapter
The thalamus is a key structure in the mammalian brain, providing a hub for communication within and across distributed forebrain networks. Research in this area has undergone a revolution in the last decade, with findings that suggest an expanded role for the thalamus in sensory processing, motor control, arousal regulation, and cognition. Moving...
Article
A hallmark of human intelligence is the ability to adapt to new situations by applying learned rules to new content (systematicity) and thereby enabling an open-ended number of inferences and actions (generativity). Here, we propose that the human brain accomplishes these feats through pathways in the parietal cortex that encode the abstract struct...
Preprint
Full-text available
Neural networks struggle in continual learning settings from catastrophic forgetting: when trials are blocked, new learning can overwrite the learning from previous blocks. Humans learn effectively in these settings, in some cases even showing an advantage of blocking, suggesting the brain contains mechanisms to overcome this problem. Here, we buil...
Preprint
The hippocampus plays a critical role in the rapid learning of new episodic memories. Many computational models propose that the hippocampus is an autoassociator that relies on Hebbian learning (i.e., “cells that fire together, wire together”). However, Hebbian learning is computationally suboptimal as it modifies weights unnecessarily beyond what...
Preprint
Full-text available
Infants, adults, non-human primates and non-primates all learn patterns implicitly, and they do so across modalities. The biological evidence supports the hypothesis that the mechanism for this learning is general but computationally local. We hypothesize that the mechanism itself is predictive error-driven learning. We build on recent work that ad...
Preprint
Full-text available
A hallmark of human intelligence is the ability to adapt to new situations, by applying learned rules to new content (systematicity) and thereby enabling an open-ended number of inferences and actions (generativity). Here, we propose that the human brain accomplishes these feats through pathways in the parietal cortex that encode the abstract struc...
Article
Full-text available
The neural mechanisms supporting flexible relational inferences, especially in novel situations, are a major focus of current research. In the complementary learning systems framework, pattern separation in the hippocampus allows rapid learning in novel environments, while slower learning in neocortex accumulates small weight changes to extract sys...
Preprint
Full-text available
The neural mechanisms supporting flexible relational inferences, especially in novel situations, are a major focus of current research. In the complementary learning systems framework, pattern separation in the hippocampus allows rapid learning in novel environments, while slower learning in neocortex accumulates small weight changes to extract sys...
Article
How do humans learn from raw sensory experience? Throughout life, but most obviously in infancy, we learn without explicit instruction. We propose a detailed biological mechanism for the widely embraced idea that learning is driven by the differences between predictions and actual outcomes (i.e., predictive error-driven learning). Specifically, num...
Article
Full-text available
Compared to our understanding of positive prediction error signals occurring due to unexpected reward outcomes, less is known about the neural circuitry in humans that drives negative prediction errors during omission of expected rewards. While classical learning theories such as Rescorla–Wagner or temporal difference learning suggest that both typ...
Article
We present a theory and neural network model of the neural mechanisms underlying human decision-making. We propose a detailed model of the interaction between brain regions, under a proposer-predictor-actor-critic framework. This theory is based on detailed animal data and theories of action-selection. Those theories are adapted to serial operation...
Chapter
Episodic memory retrieval is often viewed as a static process that enables one to re-experience past events. However, there is considerable evidence suggesting that memories can be modified, altered, or strengthened through online retrieval and reactivation during sleep. For example, recent research has demonstrated the importance of testing in lea...
Article
Eye movement parameters are consistently investigated, and the distribution of these parameters are well known. Whereas saccade duration has been studied along with saccade amplitude , the distribution of saccade duration has not yet been reported. We aim to investigate the distribution of saccade duration in several eye movement datasets from the...
Article
Motor dysfunction in youth at clinical high risk (CHR) for psychosis is thought to reflect abnormal neurodevelopment within cortical-subcortical motor circuits and may be important for understanding clinical trajectories of CHR individuals. However, to date, our perspective of brain-behavior relationships has been informed solely by cross-sectional...
Preprint
Full-text available
Extrapolation -- the ability to make inferences that go beyond the scope of one's experiences -- is a hallmark of human intelligence. By contrast, the generalization exhibited by contemporary neural network algorithms is largely limited to interpolation between data points in their training corpora. In this paper, we consider the challenge of learn...
Preprint
Full-text available
How does the human brain learn new concepts from raw sensory experience, without explicit instruction? We still do not have a widely-accepted answer to this central question. Here, we propose a detailed biological mechanism for the widely-embraced idea that learning is based on the differences between predictions and actual outcomes (i.e., predicti...
Article
Full-text available
We describe a neurobiologically informed computational model of phasic dopamine signaling to account for a wide range of findings, including many considered inconsistent with the simple reward prediction error (RPE) formalism. The central feature of this PVLV framework is a distinction between a primary value (PV) system for anticipating primary re...
Article
Motivation plays a central role in human behavior and cognition but is not well captured by widely used artificial intelligence (AI) and computational modeling frameworks. This Opinion article addresses two central questions regarding the nature of motivation: what are the nature and dynamics of the internal goals that drive our motivational system...
Preprint
Full-text available
Several experiments, notably one done by Bruner and Potter (1964), have demonstrated delayed object recognition when viewing a blurred image gradually come into focus. Bruner and Potter (1964) suggested that the wrong answer is held until there is an obvious contradiction. Others have hypothesized that “competitive activation” is responsible for de...
Article
Full-text available
We address the distinction between habitual/automatic vs. goal-directed/controlled behavior, from the perspective of a computational model of the frontostriatal loops. The model exhibits a continuum of behavior between these poles, as a function of the interactive dynamics among different functionally-specialized brain areas, operating iteratively...
Preprint
Full-text available
Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution. However, human learners readily generalize in this way, e.g. by applying known grammatical rules to novel words. Inspired by work in neuroscie...
Chapter
Computational models of frontal function have made important contributions to understanding how the frontal lobes support a wide range of important functions, in their interactions with other brain areas including, critically, the basal ganglia (BG). We focus here on the specific case of how different frontal areas support goal-directed, motivated...
Article
Full-text available
How does the neocortex learn and develop the foundations of all our high-level cognitive abilities? We present a comprehensive framework spanning biological, computational, and cognitive levels, with a clear theoretical continuity between levels, providing a coherent answer directly supported by extensive data at each level. Learning is based on ma...
Article
Full-text available
One candidate approach to creating artificial general intelligence (AGI) is to imitate the essential computations of human cognition. This process is sometimes called ‘reverse-engineering the brain’ and the end product called ‘neuromorphic.’ We argue that, unlike with other approaches to AGI, anthropomorphic reasoning about behaviour and safety con...
Conference Paper
Full-text available
Learning about conditioned inhibitors, which predict omission of outcome delivery, has been relatively understudied compared to learning about reward predictors. Reward omissions lead to inhibition of dopamine neurons, driven by the lateral habenula, an important region that is also involved in learning about predictors of punishment. How could a c...
Article
Full-text available
Visual selection processes in real-world scenes are guided by exogenous and endogenous factors. Endogenous factors correspond to top-down influences like the cognitive tasks or prior knowledge. Object processing is affected by the gist of the scene within which it is embedded and the prior knowledge about the objects. On one hand, it has been shown...
Article
Full-text available
Background: Previous research in patients with anorexia nervosa showed heightened brain response during a taste reward conditioning task and heightened sensitivity to rewarding and punishing stimuli. Here we tested the hypothesis that individuals recovered from anorexia nervosa would also experience greater brain activation during this task as wel...
Article
Full-text available
Decades of animal and human neuroimaging research have identified distinct, but overlapping, striatal zones, which are interconnected with separable corticostriatal circuits, and are crucial for the organization of functional systems. Despite continuous efforts to subdivide the human striatum based on anatomical and resting-state functional connect...
Article
Full-text available
Aksentijevic (1) raises issues with our recent report (2) that bear further clarification because they reflect some fundamental confusion about the nature of random sequences. First, it is essential to distinguish statistics for individual elements (i.e., patterns of length one) vs. statistics for patterns consisting of more than one element (i.e.,...
Article
Full-text available
We present a cerebellar architecture with two main characteristics. The first one is that complex spikes respond to increases in sensory errors. The second one is that cerebellar modules associate particular contexts where errors have increased in the past with corrective commands that stop the increase in error. We analyze our architecture formall...
Article
Full-text available
People generally fail to produce random sequences by overusing alternating patterns and avoiding repeating ones-the gambler's fallacy bias. We can explain the neural basis of this bias in terms of a biologically motivated neural model that learns from errors in predicting what will happen next. Through mere exposure to random sequences over time, t...
Article
Full-text available
The development of intravenous fat emulsion (IVFE) is the culmination of physiological, biochemical, nutritional, and medical scientific advancements. IVFEs have the ability to deliver critical nutritional substrates to the patient. Recent literature purports that they may also play roles in modulation of immune functionality and pulmonary physiolo...
Article
Full-text available
Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, o...
Article
As focus shifts to large-scale network interactions involved in memory, it is becoming increasingly clear that oscillatory dynamics are critically involved. A number of studies have shown a negative correlation between memory retrieval in alpha and beta power, and a positive correlation between retrieval and theta power. In this opinion article, we...
Article
Full-text available
We present a model of the cerebellum with two defining characteristics. The first one is that complex spikes respond to increases in sensory errors. The second one is that cerebellar modules implement forward models that predict how a controller will react to reduce error, instead of predicting the consequence of motor commands. We examine the biol...
Article
Full-text available
We present a comprehensive, novel framework for understanding how the neocortex, including the thalamocortical loops through the deep layers, can support a temporal context representation in the service of predictive learning. Many have argued that predictive learning provides a compelling, powerful source of learning signals to drive the developme...
Article
Full-text available
Standard models of the visual object recognition pathway hold that a largely feedforward process from the retina through inferotemporal cortex leads to object identification. A subsequent feedback process originating in frontoparietal areas through reciprocal connections to striate cortex provides attentional support to salient or behaviorally-rele...
Article
Full-text available
Current theoretical and computational models of dopamine-based reinforcement learning are largely rooted in the classical behaviorist tradition, and envision the organism as a purely reactive recipient of rewards and punishments, with resulting behavior that essentially reflects the sum of this reinforcement history. This framework is missing some...
Article
The Synthesis of ACT-R and Leabra (SAL) hybrid cognitive architecture is the integration of two theories of cognitive functioning, each itself a highly integrative theory of cognition, ACT-R being predominantly a symbolic production-rule based architecture and Leabra a neural modeling architecture. The combination of the two architectures allows fo...
Article
Cognitive actions are frequently taken with the expectation of a particular result in mind. The expectation is generally based on past experiences, but not necessarily on any specific experience in isolation; rather, the integrated collection of past experiences forms the basis for what we should expect. Although an important part of action executi...
Article
Full-text available
The ability to flexibly, rapidly, and accurately perform novel tasks is a hallmark of human behavior. In our everyday lives we are often faced with arbitrary instructions that we must understand and follow, and we are able to do so with remarkable ease. It has frequently been argued that this ability relies on symbol processing, which depends criti...
Article
Control processes are critical for both facilitating and suppressing memory retrieval, but these processes are not well understood. The current work, inspired by a similar fMRI design [8], used a modified Think/No-Think(TNT) paradigm to investigate the neural signatures of volition over enhancing and suppressing memory retrieval. Previous studies h...
Article
Full-text available
The learning mechanism in the hippocampus has almost universally been assumed to be Hebbian in nature, where individual neurons in an engram join together with synaptic weight increases to support facilitated recall of memories later. However, it is also widely known that Hebbian learning mechanisms impose significant capacity constraints, and are...
Article
Full-text available
How does the brain learn to recognize objects visually, and perform this difficult feat robustly in the face of many sources of ambiguity and variability? We present a computational model based on the biology of the relevant visual pathways that learns to reliably recognize 100 different object categories in the face of naturally occurring variabil...
Article
Full-text available
We can learn from the wisdom of others to maximize success. However, it is unclear how humans take advice to flexibly adapt behavior. On the basis of data from neuroanatomy, neurophysiology, and neuroimaging, a biologically plausible model is developed to illustrate the neural mechanisms of learning from instructions. The model consists of two comp...
Article
Full-text available
We address strategic cognitive sequencing, the "outer loop" of human cognition: how the brain decides what cognitive process to apply at a given moment to solve complex, multistep cognitive tasks. We argue that this topic has been neglected relative to its importance for systematic reasons but that recent work on how individual brain systems accomp...
Article
The Synthesis of ACT-R and Leabra (SAL) hybrid cognitive architecture is the integration of two theories of cognitive functioning, each itself a highly integrative theory of cognition, ACT-R being predominantly a symbolic production-rule based architecture and Leabra a neural modeling architecture. The combination of the two architectures allows fo...
Article
Full-text available
When we behave according to rules and instructions, our brains interpret abstract representations of what to do and transform them into actual behavior. In order to investigate the neural mechanisms behind this process, we devised an fMRI experiment that explicitly isolated rule interpretation from rule encoding and execution. Our results showed th...
Article
Full-text available
Everyday vision requires robustness to a myriad of environmental factors that degrade stimuli. Foreground clutter can occlude objects of interest, and complex lighting and shadows can decrease the contrast of items. How does the brain recognize visual objects despite these low-quality inputs? On the basis of predictions from a model of object recog...
Article
Full-text available
How does the brain bind together visual features that are processed concurrently by different neurons into a unified percept suitable for processes such as object recognition? Here, we describe how simple, commonly accepted principles of neural processing can interact over time to solve the brain's binding problem. We focus on mechanisms of neural...
Article
Full-text available
Evidence suggests that two regions of the striatum contribute differential support to instrumental response selection. The dorsomedial striatum (DMS) is thought to support expectancy-mediated actions, and the dorsolateral striatum (DLS) is thought to support habits. Currently it is unclear whether these regions store task-relevant information or ju...
Article
Full-text available
Anorexia nervosa (AN) is a severe psychiatric disorder associated with food avoidance and malnutrition. In this study, we wanted to test whether we would find brain reward alterations in AN, compared with individuals with normal or increased body weight. We studied 21 underweight, restricting-type AN (age M 22.5, SD 5.8 years), 19 obese (age M 27.1...
Article
Full-text available
The present experiment tested three hypotheses regarding the function and organization of lateral prefrontal cortex (PFC). The first account (the information cascade hypothesis) suggests that the anterior-posterior organization of lateral PFC is based on the timing with which cue stimuli reduce uncertainty in the action selection process. The secon...