Blake Richards

Blake Richards
University of Toronto | U of T · Department of Biological Sciences at Scarborough

About

66
Publications
29,497
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
2,637
Citations

Publications

Publications (66)
Preprint
Full-text available
To unveil how the brain learns, ongoing work seeks biologically-plausible approximations of gradient descent algorithms for training recurrent neural networks (RNNs). Yet, beyond task accuracy, it is unclear if such learning rules converge to solutions that exhibit different levels of generalization than their nonbiologically-plausible counterparts...
Article
Full-text available
It is commonly assumed that usage of the word “computer” in the brain sciences reflects a metaphor. However, there is no single definition of the word “computer” in use. In fact, based on the usage of the word “computer” in computer science, a computer is merely some physical machinery that can in theory compute any computable function. According t...
Preprint
Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains. Consequently, the notion of applying learning methods to a particular problem set has become an established and valuable modus operandi to advance a particular field. In this article we argue that such an approach does no...
Article
Full-text available
The connection patterns of neural circuits in the brain form a complex network. Collective signalling within the network manifests as patterned neural activity and is thought to support human cognition and adaptive behaviour. Recent technological advances permit macroscale reconstructions of biological brain networks. These maps, termed connectomes...
Article
Full-text available
Neurons can carry information with both the synchrony and rate of their spikes. However, it is unknown whether distinct subtypes of neurons are more sensitive to information carried by synchrony versus rate, or vice versa. Here, we address this question using patterned optical stimulation in slices of somatosensory cortex from mouse lines labelling...
Article
Neurons are very complicated computational devices, incorporating numerous non-linear processes, particularly in their dendrites. Biophysical models capture these processes directly by explicitly modelling physiological variables, such as ion channels, current flow, membrane capacitance, etc. However, another option for capturing the complexities o...
Preprint
Full-text available
Neurons in the dorsal visual pathway of the mammalian brain are selective for motion stimuli, with the complexity of stimulus representations increasing along the hierarchy. This progression is similar to that of the ventral visual pathway, which is well characterized by artificial neural networks (ANNs) optimized for object recognition. In contras...
Article
Full-text available
Synaptic plasticity is believed to be a key physiological mechanism for learning. It is well established that it depends on pre- and postsynaptic activity. However, models that rely solely on pre- and postsynaptic activity for synaptic changes have, so far, not been able to account for learning complex tasks that demand credit assignment in hierarc...
Preprint
Full-text available
This perspective piece came about through the Generative Adversarial Collaboration (GAC) series of workshops organized by the Computational Cognitive Neuroscience (CCN) conference in 2020. We brought together a number of experts from the field of theoretical neuroscience to debate emerging issues in our understanding of how learning is implemented...
Article
Full-text available
What is the purpose of dreaming? Many scientists have postulated a role for dreaming in learning, often with the aim of improving generative models. In this issue of Patterns, Erik Hoel proposes a novel hypothesis, namely, that dreaming provides a means to reduce overfitting. This hypothesis is interesting both for neuroscience and for the developm...
Preprint
Full-text available
Neurons are very complicated computational devices, incorporating numerous non-linear processes, particularly in their dendrites. Biophysical models capture these processes directly by explicitly modelling physiological variables, such as ion channels, current flow, membrane capacitance, etc. However, another option for capturing the complexities o...
Preprint
Full-text available
Dynamic latent variable modelling has provided a powerful tool for understanding how populations of neurons compute. For spiking data, such latent variable modelling can treat the data as a set of point-processes, due to the fact that spiking dynamics occur on a much faster timescale than the computational dynamics being inferred. In contrast, for...
Preprint
Full-text available
Scientists have long conjectured that the neocortex learns the structure of the environment in a predictive, hierarchical manner. To do so, expected, predictable features are differentiated from unexpected ones by comparing bottom-up and top-down streams of data. It is theorized that the neocortex then changes the representation of incoming stimuli...
Article
Behavioral flexibility is important in a changing environment. Previous research suggests that systems consolidation, a long-term poststorage process that alters memory traces, may reduce behavioral flexibility. However, exactly how systems consolidation affects flexibility is unknown. Here, we tested how systems consolidation affects: (1) flexibil...
Article
Full-text available
Synchronization of precise spike times across multiple neurons carries information about sensory stimuli. Inhibitory interneurons are suggested to promote this synchronization, but it is unclear whether distinct interneuron subtypes provide different contributions. To test this, we examined single-unit recordings from barrel cortex in vivo and used...
Preprint
Full-text available
Synaptic plasticity is believed to be a key physiological mechanism for learning. It is well-established that it depends on pre and postsynaptic activity. However, models that rely solely on pre and postsynaptic activity for synaptic changes have, to date, not been able to account for learning complex tasks that demand hierarchical networks. Here,...
Article
Full-text available
Background: Abnormal accumulation of amyloid β1-42 oligomers (AβO1-42), a hallmark of Alzheimer's disease, impairs hippocampal theta-nested gamma oscillations and long-term potentiation (LTP) that are believed to underlie learning and memory. Parvalbumin-positive (PV) and somatostatin-positive (SST) interneurons are critically involved in theta-ne...
Article
Systems neuroscience seeks explanations for how the brain implements a wide variety of perceptual, cognitive and motor tasks. Conversely, artificial intelligence attempts to design computational systems based on the tasks they will have to solve. In artificial neural networks, the three components specified by design are the objective functions, th...
Preprint
Full-text available
In artificial neural networks trained with gradient descent, the weights used for processing stimuli are also used during backward passes to calculate gradients. For the real brain to approximate gradients, gradient information would have to be propagated separately, such that one set of synaptic weights is used for processing and another set is us...
Article
Neurogenesis persists throughout life in the dentate gyrus region of the mammalian hippocampus. Computational models have established that the addition of neurons degrades existing memories (i.e., produces forgetting). These predictions are supported by empirical observations in rodents, where post-training increases in neurogenesis also promote fo...
Preprint
Full-text available
Synchronization of precise spike-times across multiple neurons carries information about sensory stimuli. Inhibitory interneurons are suggested to promote this synchronization, but it is unclear whether distinct interneuron subtypes provide different contributions. To test this, we examined single-unit recordings from barrel cortex in vivo and used...
Preprint
Full-text available
Populations of neurons in the neocortex can carry information with both the synchrony and the rate of their spikes. However, it is unknown whether distinct subtypes of neurons in the cortical microcircuit are more sensitive to information carried by synchrony versus rate. Here, we address this question using patterned optical stimulation in slices...
Article
Full-text available
Symptoms of schizophrenia may arise from a failure of cortical circuits to filter-out irrelevant inputs. Schizophrenia has also been linked to disruptions in cortical inhibitory interneurons, consistent with the possibility that in the normally functioning brain, these cells are in some part responsible for determining which sensory inputs are rele...
Data
Demonstration of learning to ignore with multiple inhibitory units (500 inhibitory units). (A) Average Cortex excitatory unit activity (lower plots) and average cortex inhibitory unit activity (upper plots) at simulated, 20 ms time steps in response to unlearned stimuli (left side) compared with the end of a series of repeated presentations (right...
Data
Multiplexed stimulus category and relevance codes with multiple rewarded stimuli. (A) Diagram illustrating modified model that included both the mechanisms described above for relevance learning (on Wx→I synapses) in addition to mechanisms learning an output vector that matches categories presented as input (backpropagation algorithm applied to the...
Data
Learned irrelevance—Slowed relevance learning following uncorrelated CS-US presentations. (A) Average excitatory unit responses to each presentation of a CS in a learned irrelevance paradigm. One network (red) was exposed to 100 presentations of a CS and an US, where CS and US presentation times were chosen from independent uniform distributions, f...
Data
Code to run simulations. Matlab code for running the simulations that generated the data presented in the paper is provided here. Note that the code utilizes the Statistics Toolbox. To run a custom simulation, refer to main_script.m. In order to reproduce any of the figures in the paper, simply run one of the following files instead: learning_to_ig...
Data
Learning with delays between CS+ offset and US onset. (A) Average Cortex excitatory unit activity (lower plots) and inhibitory unit activity (upper plots) when the offset of CS+ precedes the onset of the US by 100 ms (note the gap between the green and gray blocks at the bottom). This effect was achieved by using γ = 0.98 and an eligibility trace f...
Data
Firing-rate and weight distributions following learning to ignore training. (A) Firing-rate distributions across the E(t) population during the simulation. Time-bins were 200 ms long. (B) Synaptic weight distributions for the Wx→I weights following learning to ignore training. (TIFF)
Preprint
Full-text available
The backpropagation of error algorithm (BP) is often said to be impossible to implement in a real brain. The recent success of deep networks in machine learning and AI, however, has inspired proposals for understanding how the brain might learn across multiple layers, and hence how it might implement or approximate BP. As of yet, none of these prop...
Article
Full-text available
Deep learning has led to significant advances in artificial intelligence, in part, by adopting strategies motivated by neurophysiology. However, it is unclear whether deep learning could occur in the real brain. Here, we show that a deep learning algorithm that utilizes multi-compartment neurons might help us to understand how the neocortex optimiz...
Data
Fig_8B_errors.csv. This data file contains the test error (measured on 10,000 MNIST images not used for training) across 60 epochs of training, for our standard one hidden layer network (Regular) and a network with sparse feedback weights. Fig_8B_final_errors.csv. This data file contains the results of repeated weight tests (n=20) after 60 epochs f...
Data
Fig_6B_errors.csv. This data file contains the test error (measured on 10,000 MNIST images not used for training) across 60 epochs of training, for a network with no hidden layers, a network with one hidden layer, and a network with two hidden layers. Fig_6B_final_errors.csv. This data file contains the results of repeated weight tests (n=20) after...
Data
Fig_9B_errors.csv. This data file contains the test error (measured on 10,000 MNIST images not used for training) across 60 epochs of training, for a two hidden layer network, with total apical segregation (Regular), strong apical attenuation and weak apical attenuation. Fig_9B_final_errors.csv. This data file contains the results of repeated weigh...
Data
Fig_7A.csv. This data file contains the time-averaged angle (with a sliding window of 100 images) between weight updates prescribed by our local update learning algorithm compared to those prescribed by backpropagation of error, for a one hidden layer network over 10 epochs of training (600,000 training examples). Fig_7C.csv. The first column of th...
Data
Fig_5B.csv. The first two columns of the data file contain the hidden layer loss (L0) and output layer loss (L1) of a one hidden layer network in response to all ‘2’ images in the MNIST test set after one epoch of training. The last two columns contain the same data, except that the data in the third column (Shuffled data L0) was generated by rando...
Article
Full-text available
Following learning, increased coupling between spindle oscillations in the medial prefrontal cortex (mPFC) and ripple oscillations in the hippocampus is thought to underlie memory consolidation. However, whether learning-induced increases in ripple-spindle coupling are necessary for successful memory consolidation has not been tested directly. In o...
Preprint
Full-text available
Symptoms of schizophrenia may arise from a failure of cortical circuits to filter-out irrelevant inputs. Schizophrenia has also been linked to disruptions to cortical inhibitory interneurons, consistent with the possibility that in the normally functioning brain, these cells are in some part responsible for determining which inputs are relevant and...
Article
The predominant focus in the neurobiological study of memory has been on remembering (persistence). However, recent studies have considered the neurobiology of forgetting (transience). Here we draw parallels between neurobiological and computational mechanisms underlying transience. We propose that it is the interaction between persistence and tran...
Article
Full-text available
Deep learning has led to significant advances in artificial intelligence in recent years, in part by adopting architectures and functions motivated by neurophysiology. However, current deep learning algorithms are biologically infeasible, because they assume non-spiking units, discontinuous-time, and non-local synaptic weight updates. Here, we buil...
Article
The ability to generate action potentials (spikes) in response to synaptic input determines whether a neuron participates in information processing. How a developing neuron becomes an active participant in a circuit or whether this process is activity dependent is not known, especially as spike-dependent plasticity mechanisms would not be available...
Article
Memories are thought to be sparsely encoded in neuronal networks, but little is known about why a given neuron is recruited or allocated to a particular memory trace. Previous research shows that in the lateral amygdala (LA), neurons with increased CREB are selectively recruited to a fear memory trace. CREB is a ubiquitous transcription factor impl...
Article
Full-text available
Understanding how neurons acquire specific response properties is a major goal in neuroscience. Recent studies in mouse neocortex have shown that "sister neurons" derived from the same cortical progenitor cell have a greater probability of forming synaptic connections with one another [1, 2] and are biased to respond to similar sensory stimuli [3,...
Article
Memories are not static but continue to be processed after encoding. This is thought to allow the integration of related episodes via the identification of patterns. Although this idea lies at the heart of contemporary theories of systems consolidation, it has yet to be demonstrated experimentally. Using a modified water-maze paradigm in which plat...
Article
Full-text available
Throughout life, new neurons are continuously added to the dentate gyrus. As this continuous addition remodels hippocampal circuits, computational models predict that neurogenesis leads to degradation or forgetting of established memories. Consistent with this, increasing neurogenesis after the formation of a memory was sufficient to induce forgett...
Article
Memories serve to establish some permanence to our inner lives despite the fleeting nature of subjective experience. Most neurobiological theories of memory assume that this mental permanence reflects an underlying cellular permanence. Namely, it is assumed that the cellular changes which first occur to store a memory are perpetuated for as long as...
Article
Full-text available
Memory stabilization following encoding (synaptic consolidation) or memory reactivation (reconsolidation) requires gene expression and protein synthesis (Dudai and Eisenberg, 2004; Tronson and Taylor, 2007; Nader and Einarsson, 2010; Alberini, 2011). Although consolidation and reconsolidation may be mediated by distinct molecular mechanisms (Lee et...
Article
The retinotectal pathway of Xenopus laevis is a well-established experimental model for studying activity-dependent processes during visual system development. Such processes can be guided by stimulus-evoked activity patterns, which depend on the refractive characteristics of the eye. Previous work has shown that many animals are hyperopic at early...
Article
Full-text available
We argue that an explanation of relevance realization is a pervasive problem within cognitive science, and that it is becoming the criterion of the cognitive in terms of which a new framework for doing cognitive science is emerging. We articulate that framework and then make use of it to provide the beginnings of a theory of relevance realization t...
Article
Full-text available
During the development of sensory systems, receptive fields are modified by stimuli in the environment. This is thought to rely on learning algorithms that are sensitive to correlations in spike timing between cells, but the manner in which developing circuits selectively exploit correlations that are related to sensory inputs is unknown. We record...
Article
Full-text available
Spike-timing-dependent plasticity (STDP) is found in vivo in a variety of systems and species, but the first demonstrations of in vivo STDP were carried out in the optic tectum of Xenopus laevis embryos. Since then, the optic tectum has served as an excellent experimental model for studying STDP in sensory systems, allowing researchers to probe the...
Article
Full-text available
Abnormalities in cognitive function and brain structure have been reported in acutely ill adolescents with anorexia nervosa, but whether these abnormalities persist or are reversible in the context of weight restoration remains unclear. Brain structure and cognitive function in female subjects with adolescent-onset anorexia nervosa assessed at long...
Article
In vivo measurement of cortical thickness is a sensitive representation of pathology in neurodegenerative disorders which primarily target the gray mantle. In this study we used magnetic resonance images to describe the patterns of cortical thinning in 11 frontotemporal dementia (FTD), 38 Alzheimer's disease (AD) and 34 healthy elderly (H(E)) subje...
Article
The present investigation sought to identify which brain regions distinguish pedophilic from nonpedophilic men, using unbiased, automated analyses of the whole brain. T1-weighted magnetic resonance images (MRIs) were acquired from men who demonstrated illegal or clinically significant sexual behaviors or interests (n = 65) and from men who had hist...
Article
Full-text available
Children learn to name the objects they see by forming general associations between the words they hear and the images arriving at their retina. Discriminative neural network models can also be taught to classify objects, but to do so they require more information about how images pair with words (i.e. supervised data) than the brain seems to recei...
Article
The objectives of this research were to further delineate the neural circuits subserving proposed memory-based behavioural subsystems in the hippocampal formation. These studies were guided by anatomical evidence showing a topographical organization of the hippocampal formation. Briefly, perpendicular to the medial/lateral entorhinal cortex divisio...

Projects

Projects (2)
Project
To understand how cost function optimization is coordinated across different regions of the neocortex.