Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Network architecture and baseline dynamics. (A) Basic network architecture (PY: excitatory pyramidal neurons; IN: inhibitory interneurons; TC: excitatory thalamocortical neurons; RE: inhibitory thalamic reticular neurons). Excitatory synapses are represented by lines terminating in a dot, while inhibitory synapses are represented by lines terminating in bars. Arrows indicate the direction of the connection. (B) Behavior of a control network exhibiting wake-sleep transitions. Cortical PY neurons are shown. Color represents the voltage of a neuron at a given time during the simulation (dark blue -hyperpolarized potential; light blue / yellow -depolarized potential; red -spike). (C) Zoom-in of a subset of neurons from the network in B (time is indicated by arrows). Left and right panels show spontaneous activity during awake-like state before and after sleep, respectively. Middle panel shows example of activity during sleep. (D) Left panel shows the initial weighted adjacency matrix for the network in B. The color in this plot represents the strength of the AMPA connections between PY neurons, with white indicating the lack of synaptic connection. Right panel shows the initial weighted adjacency matrix for the subregion indicated on the left.
Source publication
Continual learning remains an unsolved problem in artificial neural networks. The brain has evolved mechanisms to prevent catastrophic forgetting of old knowledge during new training. Building upon data suggesting the importance of sleep in learning and memory, we tested a hypothesis that sleep protects old memories from being forgotten after new l...
Contexts in source publication
Context 1
... network model, used in our study, represents a minimal thalamocortical architecture implementing one cortical layer (consisting of excitatory pyramidal (PY) and inhibitory (IN) neurons) and one thalamic layer (consisting of excitatory thalamic relay (TC) and inhibitory reticular thalamic (RE) neurons) -with all neurons simulated by Hodgkin-Huxley models ( Figure 1A). These models were built upon neuron models we used in our earlier work Wei et al., 2016;Wei et al., 2018). ...
Context 2
... the thalamic population was part of the network, its role was limited to help simulate realistic Up and Down state activity ( Bazhenov et al., 2002), as all changes occurred in the cortical population. The initial strength of the synaptic connections between cortical PY neurons was Gaussian distributed ( Figure 1D). ...
Context 3
... the following we looked separately at these two classes. We also compared synaptic weights dynamics during sleep (Figure 7) vs interleaved training (Figure 7-figure supplement 1). ...
Context 4
... doing so, sleep effectively converted recurrent connections into unidirectional connections which preferentially contributed to one memory sequence or another. It should be noted that interleaved training resulted in similar separation of weights such that some previously recurrent synapses became functionally unidirectional (Figure 7-figure supplement 1A, B). Interleaved training, however, retained more recurrent weights than sleep likely contributing to the smaller improvement in performance during post-interleaved training testing (Figure 4-figure supplement 1C). ...
Context 5
... the network was comprised of a thalamus which contained 100 thalamocortical relay neurons (TC) and 100 reticular neurons (RE), and a cortex containing 500 pyramidal neurons (PY) and 100 inhibitory interneurons (IN). The model contained only local network connectivity as described in Figure 1. Excitatory synaptic connections were mediated by AMPA and NMDA connections, while inhibitory synapses were mediated through GABA A and GABA B . ...
Similar publications
Continual learning remains an unsolved problem in artificial neural networks. The brain has evolved mechanisms to prevent catastrophic forgetting of old knowledge during new training. Building upon data suggesting the importance of sleep in learning and memory, we tested a hypothesis that sleep protects old memories from being forgotten after new l...
Citations
... However, the brain continuously forms new memories, raising the question of how new memories are integrated without disrupting previously formed ones. This challenge, termed "catastrophic forgetting" in artificial neural networks (González et al., 2020;McNaughton, 2010), arises when new information overwrites existing memories. In contrast, biological systems exhibit remarkable resilience, enabling continual learning without significant loss of prior knowledge. ...
... In their model that simulated REM sleep, the neocortex was allowed to operate with no hippocampal influence, and replay focused on repairing the old, related information to allow for "graceful continual learning." Gonzalez and colleagues (2020) developed a biophysical model of thalamocortical architecture to examine how multiple competing memories can be reinstated during NREM to prevent catastrophic forgetting and that the dynamics of REM sleep could leveraged to rescue damaged memories from interference 173 . Together, these models suggest potential underlying mechanisms of REM sleep that refine memory representations and rescue weaker memories damaged by interference or age. ...
Despite extensive evidence on the roles of non-rapid eye movement (NREM) and REM sleep in memory processing, a comprehensive model that integrates their complementary functions remains elusive due to a lack of mechanistic understanding of REM’s role in offline memory processing. We present the REM Refining and Rescuing (RnR) Hypothesis, which posits that the principal function of REM sleep is to increase the signal-to-noise ratio within and across memory representations. As such, REM sleep selectively enhances essential nodes within a memory representation while inhibiting the majority (Refine). Additionally, REM sleep modulates weak and strong memory representations so they fall within a similar range of recallability (Rescue). Across multiple NREM-REM cycles, tuning functions of individual memory traces get sharpened, allowing for integration of shared features across representations. We hypothesize that REM sleep’s unique cellular, neuromodulatory, and electrophysiological milieu, marked by greater inhibition and a mixed autonomic state of both sympathetic and parasympathetic activity, underpins these processes. The RnR Hypothesis offers a unified framework that explains diverse behavioral and neural outcomes associated with REM sleep, paving the way for future research and a more comprehensive model of sleep-dependent cognitive functions.
... However, as knowledge and tasks are provided incrementally over a lifetime, this term is often used interchangeably with LL, with many studies not making a strict distinction between the two 5,7,21 . Several biological processes in the brain serve as the basis for this capability; For example at the macroscopic level, the states of sleep and wakefulness are controlled by the interactions of the cholinergic and histaminergic neuromodulatory systems, promoting dynamic memory storage for continuous learning 26,27 At the mesoscopic level, metaplasticity processes mediated by neuromodulators and glial cells further enhance continuous learning by selectively modulating synaptic connections 4,11 . Similarly, dendritic spike-dependent plasticity also plays a critical role by selectively preserving and updating essential synaptic connections between neurons 28,29 . ...
Recent progress in artificial intelligence (AI) has been driven by insights from neuroscience, particularly with the development of artificial neural networks (ANNs). This has significantly enhanced the replication of complex cognitive tasks such as vision and natural language processing. Despite these advances, ANNs struggle with continual learning, adaptable knowledge transfer, robustness, and resource efficiency - capabilities that biological systems handle seamlessly. Specifically, ANNs often overlook the functional and morphological diversity of the brain, hindering their computational capabilities. Furthermore, incorporating cell-type specific neuromodulatory effects into ANNs with neuronal heterogeneity could enable learning at two spatial scales: spiking behavior at the neuronal level, and synaptic plasticity at the circuit level, thereby potentially enhancing their learning abilities. In this article, we summarize recent bio-inspired models, learning rules and architectures and propose a biologically-informed framework for enhancing ANNs. Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors and dendritic compartments to simulate morphological and functional diversity of neuronal computations. Finally, we outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, balances bioinspiration and complexity, and provides scalable solutions for pressing AI challenges, such as continual learning, adaptability, robustness, and resource-efficiency.
... Two critical components which are believed to underlie memory consolidation during sleep are spontaneous replay of memory traces and local unsupervised synaptic plasticity that restricts synaptic changes to relevant memories only. During sleep, replay of recently learned memories along with relevant old memories enables the network to form stable long-term memory representations (Rasch and Born 2013) and reduces competition between memories (González et al. 2020;Golden et al. 2022). The idea of replay has been explored in machine learning to enable continual learning. ...
The performance of artificial neural networks (ANNs) degrades when training data are limited or imbalanced. In contrast, the human brain can learn quickly from just a few examples. Here, we investigated the role of sleep in improving the performance of ANNs trained with limited data on the MNIST and Fashion MNIST datasets. Sleep was implemented as an unsupervised phase with local Hebbian type learning rules. We found a significant boost in accuracy after the sleep phase for models trained with limited data in the range of 0.5-10% of total MNIST or Fashion MNIST datasets. When more than 10% of the total data was used, sleep alone had a slight negative impact on performance, but this was remedied by fine-tuning on the original data. This study sheds light on a potential synaptic weight dynamics strategy employed by the brain during sleep to enhance memory performance when training data are limited or imbalanced.
... (These outputs represent the high-level sensory representations activated by hippocampal pattern completion, via return projections to the sensory cortex.) The noise input to the autoassociative network could potentially represent random activation during sleep [138][139][140] . Attributes such as reward salience might also influence which memories are replayed but were not modelled here 141 . ...
Episodic memories are (re)constructed, share neural substrates with imagination, combine unique features with schema-based predictions and show schema-based distortions that increase with consolidation. Here we present a computational model in which hippocampal replay (from an autoassociative network) trains generative models (variational autoencoders) to (re)create sensory experiences from latent variable representations in entorhinal, medial prefrontal and anterolateral temporal cortices via the hippocampal formation. Simulations show effects of memory age and hippocampal lesions in agreement with previous models, but also provide mechanisms for semantic memory, imagination, episodic future thinking, relational inference and schema-based distortions including boundary extension. The model explains how unique sensory and predictable conceptual elements of memories are stored and reconstructed by efficiently combining both hippocampal and neocortical systems, optimizing the use of limited hippocampal storage for new and unusual information. Overall, we believe hippocampal replay training generative models provides a comprehensive account of memory construction, imagination and consolidation.
... Much of this work focuses on consolidation via hippocampal replay. Prior work has proposed that replay (or similar mechanisms) can prolong memory lifetimes (Shaham et al., 2021 ;Remme et al., 2021 ), alleviate the problem of catastrophic forgetting of previously learned information (van de Ven et al., 2020 ; González et al., 2020 ;Shaham et al., 2021 ), and facilitate generalization of learned information (McClelland et al., 1995 ;Sun et al., 2021 ). One prior theoretical study (Roxin and Fusi, 2013 ), which uses replayed activity to consolidate synaptic changes from short to long-term modules, explored how systems consolidation extends forgetting curves. ...
In a variety of species and behavioral contexts, learning and memory formation recruits two neural systems, with initial plasticity in one system being consolidated into the other over time. Moreover, consolidation is known to be selective; that is, some experiences are more likely to be consolidated into long-term memory than others. Here, we propose and analyze a model that captures common computational principles underlying such phenomena. The key component of this model is a mechanism by which a long-term learning and memory system prioritizes the storage of synaptic changes that are consistent with prior updates to the short-term system. This mechanism, which we refer to as recall-gated consolidation, has the effect of shielding long-term memory from spurious synaptic changes, enabling it to focus on reliable signals in the environment. We describe neural circuit implementations of this model for different types of learning problems, including supervised learning, reinforcement learning, and autoassociative memory storage. These implementations involve learning rules modulated by factors such as prediction accuracy, decision confidence, or familiarity. We then develop an analytical theory of the learning and memory performance of the model, in comparison to alternatives relying only on synapse-local consolidation mechanisms. We find that recall-gated consolidation provides significant advantages, substantially amplifying the signal-to-noise ratio with which memories can be stored in noisy environments. We show that recall-gated consolidation gives rise to a number of phenomena that are present in behavioral learning paradigms, including spaced learning effects, task-dependent rates of consolidation, and differing neural representations in short- and long-term pathways.
... (These outputs represent the high-level sensory representations activated by hippocampal pattern completion, via return projections to sensory cortex.) The noise input to the autoassociative network could potentially represent random activation during sleep (González et al., 2020;Pezzulo et al., 2021;Stella et al., 2019). In other words, random inputs to the hippocampus result in the reactivation of memories, and this reactivation results in consolidation. ...
Human episodic memories are (re)constructed, combining unique features with schema-based predictions, and share neural substrates with imagination. They also show systematic schema-based distortions that increase with consolidation. Here we present a computational model in which hippocampal replay (from an autoassociative network) trains generative models (variational autoencoders) in neocortex to (re)create sensory experiences via latent variable representations. Simulations using large image datasets reflect the effects of memory age and hippocampal lesions and the slow learning of statistical structure in agreement with previous models (Complementary Learning Systems and Multiple Trace Theory), but also explain schema-based distortions, imagination, inference, and continual representation learning in memory. Critically, the model suggests how unique and predictable elements of memories are stored and reconstructed by efficiently combining both hippocampal and neocortical systems, optimising the use of limited hippocampal storage. Finally, the model can be extended to sequential stimuli, including language, and multiple neocortical networks could be trained, including those with latent variable representations in entorhinal, medial prefrontal, and anterolateral temporal cortices. Overall, we believe hippocampal replay training neocortical generative models provides a comprehensive account of memory construction and consolidation.
... These changes are indexed by differences in behavior before versus after an offline period (Fig. 1b). For example, after learning a sequence of tasks, a period of sleep can repair memories that have been damaged due to interference in a preceding wakeful period [19,20,1,4] -an adaptive capacity that biologically detailed models of sleep have explored [7,22,31]. ...
A remarkable capacity of the brain is its ability to autonomously reorganize memories during offline periods. Memory replay, a mechanism hypothesized to underlie biological offline learning, has inspired offline methods for reducing forgetting in artificial neural networks in continual learning settings. A memory-efficient and neurally-plausible method is generative replay, which achieves state of the art performance on continual learning benchmarks. However, unlike the brain, standard generative replay does not self-reorganize memories when trained offline on its own replay samples. We propose a novel architecture that augments generative replay with an adaptive, brain-like capacity to autonomously recover memories. We demonstrate this capacity of the architecture across several continual learning tasks and environments.
... Catastrophic forgetting should not occur when there is no interference between the images but, as the number of overlapping pixels increases, new task training can lead to forgetting. Our studies using biophysical models of the thalamocortical network 27,28 revealed that catastrophic forgetting occurs because the network connectivity becomes dominated by the most recently learned task, so input patterns for old tasks are insufficient to activate their respective output neurons; sleep replay can redistribute the network resources more equally between tasks to allow correct recall of the old and new memories. ...
... From a neuroscience perspective, sleep reduces interference by replaying activity of recently learned tasks and old relevant (interfering) tasks 20 . Using biophysical models of brain network and testing them for simplified task of learning overlapping memory sequences, we showed that sleep replay modifies the synaptic weights to create unique synaptic representation for each task 28 . Such differential allocation of resources leads to reduced representational overlap and therefore diminishes catastrophic forgetting. ...
... Some studies suggest net reductions of synaptic weights 32 , while others argue for net increase 60 . Our work (extending ideas of ref. 28) predicts that sleep replay leads to complex reorganization of synaptic connectivity, including potentiation of some synapses and pruning of others with the goal of increasing separation between memories. We found that sleep replay may increase the contrast between memory traces by enhancing lateral inhibition, such that activation of one memory inhibits other similar memories to avoid interference 61 . ...
Artificial neural networks are known to suffer from catastrophic forgetting: when learning multiple tasks sequentially, they perform well on the most recent task at the expense of previously learned tasks. In the brain, sleep is known to play an important role in incremental learning by replaying recent and old conflicting memory traces. Here we tested the hypothesis that implementing a sleep-like phase in artificial neural networks can protect old memories during new training and alleviate catastrophic forgetting. Sleep was implemented as off-line training with local unsupervised Hebbian plasticity rules and noisy input. In an incremental learning framework, sleep was able to recover old tasks that were otherwise forgotten. Previously learned memories were replayed spontaneously during sleep, forming unique representations for each class of inputs. Representational sparseness and neuronal activity corresponding to the old tasks increased while new task related activity decreased. The study suggests that spontaneous replay simulating sleep-like dynamics can alleviate catastrophic forgetting in artificial neural networks.
... Artificial neural networks also suffer catastrophic forgetting as training for a new task causes the network to forget how to perform previous tasks (Masse et al., 2018). Several biological and artificial neural network models employ offline processing, which mimics hippocampal replay events during sleep, to reconsolidate old memory traces of previous experiences (Kirkpatrick et al., 2017;González et al., 2020;van de Ven et al., 2020;Hayes et al., 2021;Wang et al., 2022). Thus, spontaneous activity replay may bridge biological and artificial neural network studies. ...
Studying the underlying neural mechanisms of cognitive functions of the brain is one of the central questions in modern biology. Moreover, it has significantly impacted the development of novel technologies in artificial intelligence. Spontaneous activity is a unique feature of the brain and is currently lacking in many artificially constructed intelligent machines. Spontaneous activity may represent the brain's idling states, which are internally driven by neuronal networks and possibly participate in offline processing during awake, sleep, and resting states. Evidence is accumulating that the brain's spontaneous activity is not mere noise but part of the mechanisms to process information about previous experiences. A bunch of literature has shown how previous sensory and behavioral experiences influence the subsequent patterns of brain activity with various methods in various animals. It seems, however, that the patterns of neural activity and their computational roles differ significantly from area to area and from function to function. In this article, I review the various forms of the brain's spontaneous activity, especially those observed during memory processing, and some attempts to model the generation mechanisms and computational roles of such activities.
