Article

A maze learning comparison of Elman, long short-term memory, and Mona neural networks

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

This study compares the maze learning performance of three artificial neural network architectures: an Elman recurrent neural network, a long short-term memory (LSTM) network, and Mona, a goal-seeking neural network. The mazes are networks of distinctly marked rooms randomly interconnected by doors that open probabilistically. The mazes are used to examine two important problems related to artificial neural networks: (1) the retention of long-term state information and (2) the modular use of learned information. For the former, mazes impose a context learning demand: at the beginning of the maze, an initial door choice forms a context that must be remembered until the end of the maze, where the same numbered door must be chosen again in order to reach the goal. For the latter, the effect of modular and non-modular training is examined. In modular training, the door associations are trained in separate trials from the intervening maze paths, and only presented together in testing trials. All networks performed well on mazes without the context learning requirement. The Mona and LSTM networks performed well on context learning with non-modular training; the Elman performance degraded as the task length increased. Mona also performed well for modular training; both the LSTM and Elman networks performed poorly with modular training.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... For the purpose of comparison we use the standard LSTM library. To ensure that we set optimally its numerous parameters, we repeat some of the published results [117][118][119], where the performance of the LSTM model is compared to the performance of the Q-learner and Elman network. The experiments presented here are an adaptation of those proposed in literature [117][118][119]. ...
... To ensure that we set optimally its numerous parameters, we repeat some of the published results [117][118][119], where the performance of the LSTM model is compared to the performance of the Q-learner and Elman network. The experiments presented here are an adaptation of those proposed in literature [117][118][119]. The task is to learn the shortest path to the goal starting from any valid position in a discrete maze world. ...
... The original paper [117] deals with a question of context learning. Context learning demands the retention of state information for an extended duration. ...
... Three ANN models were trained and tested on the game:  Mona, a goal-seeking network [10,11].  Elman, a popular MLP recurrent network [12]. ...
... Mona [10,11] is a goal-seeking ANN that learns hierarchies of cause and effect contexts. These contexts allow Mona to predict and manipulate future events. ...
Conference Paper
Full-text available
It is well known that artificial neural networks (ANNs) can learn deterministic automata. Learning nondeterministic automata is another matter. This is important because much of the world is nondeterministic, taking the form of unpredictable or probabilistic events that must be acted upon. If ANNs are to engage such phenomena, then they must be able to learn how to deal with nondeterminism. In this project the game of Pong poses a nondeterministic environment. The learner is given an incomplete view of the game state and underlying deterministic physics, resulting in a nondeterministic game. Three models were trained and tested on the game: Mona, Elman, and Numenta's NuPIC.
... A particular maze is selected from the set as a specific task that can be discriminatively learned from experience while aided by instincts. This learning is done by a goal-seeking neural network [21] that is capable of fast trial-and-error learning with reinforcement, simulating more closely how an animal would learn from experience, rather than using conventional end-to-end backprop training. ...
... The experiential learning component was chosen to be a Mona network. A Mona network is a goal-seeking neural network that learns temporal contexts quickly based on reward [21]. The experiential learning portion of the maze task involves trial and error under the mistake-prone "supervision" of instincts, a learning environment that suits a Mona network well. ...
Article
Complex organisms exhibit both evolved instincts and experiential learning as adaptive mechanisms. In isolation, neither mechanism is sufficient to successfully navigate the environments of such organisms. Instincts provide behaviors that are generally adaptive but fail in specific cases. Learning must rely on some internal or external guidance to succeed on challenging tasks. This paper explores how instincts and experiential learning can work in tandem to solve a maze environment. Specifically, instincts comprise general knowledge of a set of related mazes representing worlds that an organism might be born into, and experiential learning discriminates specific situations in the particular maze world that an organism is born into. Synergy is accomplished by a hybrid neural network, one part instinctive and the other part capable of learning. After sufficient discriminating experiences, learning can override instinct to navigate a maze when instinct would otherwise fail. Results show a marked improvement in performance when this synergistic approach is employed relative to using either instincts or learning in isolation.
... ERNN use a type of recurrent neural network, in which one or more additional context layers storing the delayed hidden layer values are augmented [22]. With the addition of the context layer, ERNN are more effective in learning dynamic information contained in the series. ...
... For example, SARIMA model have demonstrated better performance than generalized models in forecasting cryptosporidiosis cases in northeastern Spain [62], and better than regression and decomposition models in forecasting campylobacteriosis in the United States [63], but dynamic linear models showed better performance than the SARIMA model in forecasting hepatitis A and malaria [25]. In forecasting hepatitis A cases, the traditional multilayer neural networks emerged as the better model than the ARIMA models, radial basis neural networks and time-delayed neural networks [22]. The different findings of these studies suggest that further studies focusing on the comparison of different kinds of predicting methods for different type of diseases are necessary for the application in forecasting epidemic behavior. ...
Article
Full-text available
Accurate incidence forecasting of infectious disease is critical for early prevention and for better government strategic planning. In this paper, we present a comprehensive study of different forecasting methods based on the monthly incidence of typhoid fever. The seasonal autoregressive integrated moving average (SARIMA) model and three different models inspired by neural networks, namely, back propagation neural networks (BPNN), radial basis function neural networks (RBFNN), and Elman recurrent neural networks (ERNN) were compared. The differences as well as the advantages and disadvantages, among the SARIMA model and the neural networks were summarized and discussed. The data obtained for 2005 to 2009 and for 2010 from the Chinese Center for Disease Control and Prevention were used as modeling and forecasting samples, respectively. The performances were evaluated based on three metrics: mean absolute error (MAE), mean absolute percentage error (MAPE), and mean square error (MSE). The results showed that RBFNN obtained the smallest MAE, MAPE and MSE in both the modeling and forecasting processes. The performances of the four models ranked in descending order were: RBFNN, ERNN, BPNN and the SARIMA model.
... A long-short term memory (LSTM) neural network is a type of recurrent neural network (RNN) used for time series modelling and foresting. LSTM is used to cope with the gradient disappearance and explosion of RNN networks (Hochreiter and Schmidhuber, 1997;Thomas and Portegys, 2010). ...
Article
Full-text available
This paper introduces an intelligent framework for predicting the advancing speed during earth pressure balance (EPB) shield tunnelling. Five artificial intelligence (AI) models based on machine and deep learning techniques—back-propagation neural network (BPNN), extreme learning machine (ELM), support vector machine (SVM), long-short term memory (LSTM), and gated recurrent unit (GRU)—are used. Five geological and nine operational parameters that influence the advancing speed are considered. A field case of shield tunnelling in Shenzhen City, China is analyzed using the developed models. A total of 1000 field datasets are adopted to establish intelligent models. The prediction performance of the five models is ranked as GRU > LSTM > SVM > ELM > BPNN. Moreover, the Pearson correlation coefficient (PCC) is adopted for sensitivity analysis. The results reveal that the main thrust (MT), penetration (P), foam volume (FV), and grouting volume (GV) have strong correlations with advancing speed (AS). An empirical formula is constructed based on the high-correlation influential factors and their corresponding field datasets. Finally, the prediction performances of the intelligent models and the empirical method are compared. The results reveal that all the intelligent models perform better than the empirical method.
... The Elman recurrent neural network (ERNN) is a type of recurrent neural network, wherein the context layer housing the delayed hidden layer output augments the input layer (Portegys 2010). The context layer's presence in the network makes ERNNs more efficient in modeling patterns contained in a dynamic dataset. ...
Article
Full-text available
Remotely sensed image segmentation and classification form a very important part of remote sensing which involves geodata processing and analysis. Artificial neural networks (ANNs) are powerful machine learning approaches that have been successfully implemented in numerous fields of study. There exist many kinds of neural networks and there is no single efficient approach for resolving all geospatial problems. Therefore, this research aims at investigating and evaluating the efficiency of three ANN approaches, namely, backpropagation neural network (BPNN), radial basis function neural network (RBFNN), and Elman backpropagation recurrent neural network (EBPRNN) using multi-spectral satellite images for terrain feature classification. Additionally, there has been close to no application of EBPRNN in modeling multispectral satellite images even though they also contain patterns. The efficiency of the three tested approaches is presented using the kappa coefficient, user’s accuracy, producer’s accuracy, overall accuracy, classification error, and computational simulation time. The study demonstrated that all the three ANN models achieved the aim of pattern identification, segmentation, and classification. This paper also discusses the observations of increasing sample sizes as inputs in the various ANN models. It was concluded that RBFNN’s computational time increases with increasing sample size and consequently increasing the number of hidden neurons; BPNN on overall attained the highest accuracy compared to the other models; EBPRNN’s accuracy increases with increasing sample size, hence a promising and perhaps an alternative choice to BPNN and RBFNN if very large datasets are involved. Based on the performance metrics used in this study, BPNN is the best model out of the three evaluated ANN models.
... This study is a follow-up to a previous comparison of the Mona ANN with Elman and LSTM networks using a maze-learning task (Portegys, 2010). Mona is a goal-seeking system that learns hierarchies of cause-and-effect relationships. ...
Preprint
Full-text available
This study compares the modular learning performance of two artificial neural network architectures: a Long Short-Term Memory (LSTM) recurrent network, and Morphognosis, a neural network based on a hierarchy of spatial and temporal contexts. Mazes are used to measure performance, defined as the ability to utilize independently learned mazes to solve mazes composed of them. A maze is a sequence of rooms connected by doors. The modular task is implemented as follows: at the beginning of the maze, an initial door choice forms a context that must be retained until the end of an intervening maze, where the same door must be chosen again to reach the goal. For testing, the door-association mazes and separately trained intervening mazes are presented together for the first time. While both neural networks perform well during training, the testing performance of Morphognosis is significantly better than LSTM.
... For example, in [99] the maze learning performance of LSTM is compared to two other neural network architectures. A maze is defined as a network of distinctly marked rooms randomly interconnected by doors that open probabilistically. ...
Article
Full-text available
Long Short-Term Memory (LSTM) has transformed both machine learning and neurocomputing fields. According to several online sources, this model has improved Google's speech recognition, greatly improved machine translations on Google Translate, and the answers of Amazon's Alexa. This neural system is also employed by Facebook, reaching over 4 billion LSTM-based translations per day as of 2017. Interestingly, recurrent neural networks had shown a rather discrete performance until LSTM showed up. One reason for the success of this recurrent network lies in its ability to handle the exploding / vanishing gradient problem, which stands as a difficult issue to be circumvented when training recurrent or very deep neural networks. In this paper, we present a comprehensive review that covers LSTM's formulation and training, relevant applications reported in the literature and code resources implementing this model for a toy example.
... I have previously conducted research into a number of issues that differentiate conventional AI from natural intelligence. These include context, motivation, plasticity, modularity, instinct, and surprise (Portegys 2007(Portegys , 2010(Portegys , 2013(Portegys , 2015. Morphognosis, in particular, has been previously applied to the task of nest-building by a species of pufferfish (Portegys 2019). ...
Preprint
Full-text available
A bstract Honey bees are social insects that forage for flower nectar cooperatively. When an individual forager discovers a flower patch rich in nectar, it returns to the hive and performs a “dance” in the vicinity of other bees that consists of movements communicating the direction and distance to the nectar source. The bees that receive this information then fly to the location of the nectar to retrieve it, thus cooperatively exploiting the environment. This project simulates this behavior in a cellular automaton using the Morphognosis model. The model features hierarchical spatial and temporal contexts that output motor responses from sensory inputs. Given a set of bee foraging and dancing exemplars, and exposing only the external input-output of these behaviors to the Morphognosis learning algorithm, a hive of artificial bees can be generated that forage as their biological counterparts do.
... The 'Adam' optimization algorithm was used to train the LSTM algorithm [68]. Further details of LSTM are discussed in Portegys [74], Wöllmer et al. [75], and Rivest et al. [76]. ...
Article
Full-text available
Among natural hazards, landslides are devastating in China. However, little is known regarding potential landslide-prone areas in Maoxian County. The goal of this study was to apply four deep learning algorithms, including the Convolutional Neural Network (CNN), Deep Neural Network (DNN), Long Short-Term Memory (LSTM) networks, and Recurrent Neural Network (RNN) in evaluating the possibility of landslides throughout Maoxian County, Sichuan, China. A total of 1,290 landslide records was developed using historical records, field observations, and remote sensing techniques. The landslide susceptibility maps showed that most susceptible areas were along the Minjiang River and in some parts of the southeastern portion of the study area. Slope, rainfall, and distance to faults were the most influential factors affecting landslide occurrence. Results revealed that proportion of landslide susceptible areas in Maoxian County was as follows: identified landslides (13.65–23.71%) and non-landslides (76.29–86.35%). The resultant maps were tested against known landslide locations using the Area Under the Curve (AUC). This study indicated that DNN algorithm performed better than LSTM, CNN, and RNN in identifying land-slides in Maoxian County, with AUC values (for prediction accuracy) of 87.30%, 86.50%, 85.60%, and 82.90%, respectively. The results of this study are useful for future landslide risk reduction along with devising sustainable land use planning in the study area.
... In this paper, inspired by the human memory system and the immune system, we propose a Neural-Immunology/Memory Network (NIMN), in which the aforementioned disadvantages is associated with most NN-based control methods. In this work, the Neural-Memory Network [23] [24] is presented first as a foundation, and then the superior NIMN is elaborated as we introduce the so-called 'immune selection factor'. ...
Article
Full-text available
This paper proposes a novel Neural- Immunology/Memory Network to address the problem of motion control for flapping-wing Micro Aerial Vehicles (MAVs). This network is inspired by the human memory system as well as the immune system, and it is effective in attenuating the system errors and other lumped system uncertainties. In contrast to most existing Neural Networks, the convergence of this proposed Neural- Immunology/Memory Network can be theoretically proven. Both analyses and simulations that are based on different immune factors show that the proposed control method is effective in dealing with external disturbances, system nonlinearities, uncertainties and parameter variations.
Conference Paper
In reality, the inputs of many complicated systems are continuous time-varying functions. It is difficult for traditional Elman neural networks (ENN) to simulate such complicated nonlinear systems directly because their inputs are all instantaneous constant values. To overcome this limitation, an Elman-style process neural network (EPNN) is proposed in this paper. From the point view of architecture, the EPNN is similar to the ENN. The major characteristics which distinguish the EPNN from the ENN lie in the fact that the inputs and the connection weights of the EPNN are time-varying functions. A corresponding learning algorithm based on the expansion of the orthogonal basis functions is developed. The effectiveness of the EPNN and its learning algorithm is proved by the lubricating oil iron concentration prediction in the aircraft engine health condition monitoring, and the application test results also indicate that the EPNN has a faster learning speed and a higher accuracy than the same scale ENN.
Article
Full-text available
Mona is a goal - seeking artificial neural network (ANN) that learns hierarchie s of cause and effect contexts . These contexts allow Mona to predict future events. The structure of the environment is modeled in long - term memory; the state of the environment is modeled in working memory. Mona is an active system: it uses environmental contexts to produce responses that navigate the environment toward goal events that satisfy internal needs. Goal - seeking thus also filters learning to retain more relevant information about the environment. (PDF) MONA: HIERARCHICAL CONTEXT-LEARNING IN A GOAL- SEEKING ARTIFICIAL NEURAL NETWORK. Available from: https://www.researchgate.net/publication/261698478_MONA_HIERARCHICAL_CONTEXT-LEARNING_IN_A_GOAL-_SEEKING_ARTIFICIAL_NEURAL_NETWORK [accessed Aug 31 2019].
Preprint
Full-text available
Biological neural networks operate in the presence of task disruptions as they guide organisms toward goals. A familiar stream of stimulus-response causations can be disrupted by subtask streams imposed by the environment. For example, taking a familiar path to a foraging area might be disrupted by the presence of a predator, necessitating a "detour" to the area. The detour can be a known alternative path that must be dynamically composed with the original path to accomplish the overall task. In this project, overarching base paths are disrupted by independently learned path modules in the form of insertion, substitution, and deletion modifications to the base paths such that the resulting modified paths are novel to the network. The network's performance is then tested on these paths that have been learned in piecemeal fashion. In sum, the network must compose a new task on the fly. Several network architectures are tested: Time delay neural network (TDNN), Long short-term memory (LSTM), Temporal convolutional network (TCN), and Morphognosis, a hierarchical neural network. LSTM and Morphognosis perform significantly better for this task.
Article
A multi-step forecasting method for wind speed, based on multi-attribute decision making (MADM) with empirical mode decomposition (EMD) and computational intelligence, is proposed. First, wind speed time series is decomposed into some different components using EMD theory. Second, four kinds of well trained computational intelligence algorithms, i.e., BP network, RBF network, ELMAN and support vector machine (SVM), are respectively used as forecasting models for each component. Third, subjective and objective combination weighting methods are considered together for MADM to meet deviation maximization criterion. Finally, EMD inverse transformation is used to obtain the final predicted wind speed. The prediction results show that the comprehensive forecasting model effectively reduces prediction error, and it has better generalization ability and higher precision than the above mentioned individual forecasting models.
Article
This paper addresses the problem of the crew exploration vehicle (CEV) attitude control. CEVs are NASA's next-generation human spaceflight vehicles, and they use reaction control system (RCS) jet engines for attitude adjustment, which calls for control algorithms for firing the small propulsion engines mounted on vehicles. In this work, the resultant CEV dynamics combines both actuation and attitude dynamics. Therefore, it is highly nonlinear and even coupled with significant uncertainties. To cope with this situation, a neural–immunology/memory network is proposed. It is inspired by the human memory and immune systems. The control network does not rely on precise system dynamics information. Furthermore, the overall control scheme has a simple structure and demands much less computation as compared with most existing methods, making it attractive for real-time implementation. The effectiveness of this approach is also verified via simulation.
Article
In the management of Hyriopsis Cumingii ponds, dissolved oxygen (DO) is the key point to measure, predict and control. This paper analyzes the important factors for predicting dissolved oxygen of Hyriopsis Cumingii ponds in short-term period, and finally chooses solar radiation (SR), air temperature (AT), water temperature (WT), wind speed (WS), PH and oxygen (DO) as six input parameters. As the dissolved oxygen in the outdoor pond is low controllability and scalability, this paper proposes a short-term predicting model for dissolved oxygen based on Elman neural network. Then the predicting model is trained, tested and compared with BP model. Experimental results show that: Elman neural network predicting model with good fitting ability, generalization ability, and high prediction accuracy, can better predict the short-term changes of dissolved oxygen than BP neural network. This Elman neural network model is proven to be an effective way to predict dissolved oxygen in short-term.
Article
The traditional wavelet artificial neural network (WANN) is easy to take local convergence and has slowly learning convergent velocity. Thus, a novel method based on improved wavelet neural network is proposed to industrial control industrial control network traffic prediction.In this study, networked prediction control-based traffic prediction is employed. Then.we compare the ability of artificial neural network (ANN) with improved WANN (IWANN) to the prediction of industrial control network traffic. The experiment results indicate that in the case of one-step prediction and multi-step prediction, the ANN has lower prediction precision than IWANN.Therefore, the IWANN performs a good ability in industrial control network traffic prediction.
Article
A novel fault diagnosis model for rotating machinery based on Elman neural network was proposed. By taking advantages of Empirical Mode Decomposition (EMD) in the fault feature extraction and the Elman neural network in fault pattern recognition, fault vibration signals were decomposed into several stationary Intrinsic Mode Functions(IMFs). Then, the instantaneous amplitude L2-norms of the IMFs with modulation characteristics were computed and regarded as the input characteristic vector of the trained Elman neural network for fault diagnosis. Example of deep groove ball bearing fault diagnosis validated the effectiveness of the proposed fault diagnosis model.
Article
The Cf-252 source-driven verification system (SDVS) can recognize the enrichment of fissile material with the enrichment-sensitive autocorrelation functions of a detector signal in Cf-252 source-driven noise-analysis (SDNA) measurements. We propose a parallel and optimized genetic Elman network (POGEN) to identify the enrichment of U-235 based on the physical properties of the measured autocorrelation functions. Theoretical analysis and experimental results indicate that, for 4 different enrichment fissile materials, due to higher information utilization, more efficient network architecture, and optimized parameters, the POGEN-based algorithm can obtain identification results with higher recognition accuracy, compared to the integrated autocorrelation function (IAF) method.
Conference Paper
Hyriopsis Cumingii is Chinese major fresh water pearl mussel, widely distributed in the southern provinces of China's large and medium-sized freshwater lakes. In the management of Hyriopsis Cumingii ponds, dissolved oxygen (DO) is the key point to measure, predict and control. In this study, we analyzes the important factors for predicting dissolved oxygen of Hyriopsis Cumingii ponds, and finally chooses solar radiation(SR), water temperature(WT), wind speed(WS), PH and oxygen(DO) as six input parameters. In this paper, Elman neural networks were used to predict and forecast quantitative characteristics of water. As the dissolved oxygen in the outdoor pond is low controllability and scalability, this paper proposes a predicting model for dissolved oxygen. The true power and advantage of this method lie in its ability to (1) represent both linear and non-linear relationships and (2) learn these relationships directly from the data being modeled. The study focuses on Singapore coastal waters. The Elman NN model is built for quick assessment and forecasting of selected water quality variables at any location in the domain of interest. Experimental results show that: Elman neural network predicting model with good fitting ability, generalization ability, and high prediction accuracy, can better predict the changes of dissolved oxygen.
Article
Full-text available
In this paper we present a reinforcement learning (RL) system based on neural circuits. The neural RL system is benchmarked against a Monte Carlo (MC) RL algorithm on two tasks. The first task is the classical n-armed bandit problem and the second task is path finding in a maze. The neural RL system performs equally well or better than the MC RL algorithm. The RL system presented is very flexible and general; it can easily be incorporated with other neural based systems, e.g. attractor memories.
Article
Full-text available
An important function of many organisms is the ability to learn contextual information in order to increase the probability of achieving goals. For example, a cat may watch a particular mouse hole where she has experienced success in catching mice in preference to other similar holes. Or a person will improve his chances of getting back into his house by taking his keys with him. In this paper, predisposing conditions that affect future outcomes are referred to as environmental contexts. These conditional probabilities are learned by a goal-seeking neural network. Environmental contexts of varying complexities are generated that contain conditional state-transition probabilities such that the probability of some transitions is affected by the completion of others. The neural network is capable of expressing responses that allow it to navigate the environment in order to reach a goal. The goal-seeking effectiveness of the neural network in a variety of environmental complexities is measured.
Article
Full-text available
Goal-seeking behavior in a connectionist modelis demonstrated using the examples of foragingby a simulated ant and cooperativenest-building by a pair of simulated birds. Themodel, a control neural network, translatesneeds into responses. The purpose of this workis to produce lifelike behavior with agoal-seeking artificial neural network. Theforaging ant example illustrates theintermediation of neurons to guide the ant to agoal in a semi-predictable environment. In thenest-building example, both birds, executinggender-specific networks, exhibit socialnesting and feeding behavior directed towardmultiple goals.
Conference Paper
Full-text available
Instinct and experience are shown to form a potent combination to achieve effective foraging in a simulated environment. A neural network capable of evolving instinct-related neurons and learning from experience is used as the brain of a simple foraging creature that must find food and water in a 3D block world. Instincts provide basic tactics for unsupervised exploration of the world, allowing pathways to food and water to be learned. The combination of both instinct and experience was found to be more effective than either alone. As a comparison, neural network learning also proved superior to Q-Learning on the foraging task.
Article
Full-text available
Parallel distributed processing (PDP) architectures demonstrate a potentially radical alternative to the traditional theories of language processing that are based on serial computational models. However, learning complex structural relationships in temporal data presents a serious challenge to PDP systems. For example, automata theory dictates that processing strings from a context-free language (CFL) requires a stack or counter memory device. While some PDP models have been hand-crafted to emulate such a device, it is not clear how a neural network might develop such a device when learning a CFL. This research employs standard backpropagation training techniques for a recurrent neural network (RNN) in the task of learning to predict the next character in a simple deterministic CFL (DCFL). We show that an RNN can learn to recognize the structure of a simple DCFL. We use dynamical systems theory to identify how network states reflect that structure by building counters in phase space. The work is an empirical investigation which is complementary to theoretical analyses of network capabilities, yet original in its specific configuration of dynamics involved. The application of dynamical systems theory helps us relate the simulation results to theoretical results, and the learning task enables us to highlight some issues for understanding dynamical systems that process language with counters.
Article
Full-text available
Photocopy. Supplied by British Library. Thesis (Ph. D.)--King's College, Cambridge, 1989.
Article
Full-text available
Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient-based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O(1). Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.
Article
Full-text available
Long short-term memory (LSTM; Hochreiter & Schmidhuber, 1997) can solve numerous tasks not solvable by previous learning algorithms for recurrent neural networks (RNNs). We identify a weakness of LSTM networks processing continual input streams that are not a priori segmented into subsequences with explicitly marked ends at which the network's internal state could be reset. Without resets, the state may grow indefinitely and eventually cause the network to break down. Our remedy is a novel, adaptive "forget gate" that enables an LSTM cell to learn to reset itself at appropriate times, thus releasing internal resources. We review illustrative benchmark problems on which standard LSTM outperforms other RNN algorithms. All algorithms (including LSTM) fail to solve continual versions of these problems. LSTM with forget gates, however, easily solves them, and in an elegant way.
Investigated the dependence of maze coordination upon the stability of the sensory environment in which it was developed. The method consisted of varying the environmental conditions while the maze was being learned. 200 rats were shifted from the living cage to the maze located in a different environment, thus, altering the sensory conditions. The disturbances induced by the alterations were measured in terms of the error record. Results show that sensori-motor acts cannot be regarded as an isolated independent function; the stability of the environment furthers the development of the act, and conditions the regularity and accuracy of its functioning after it has become automatic. Novel situations while running the maze may induce effects which persist and exert a disturbing influence after a return to normal conditions. Alterations of the cage environment previous to the performance of the act may exert a disturbing effect. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
This paper compares the performance of two recurrent neural network models learning the grammar of simple English sentences and introduces a method for creating useful negative training examples from a set of grammatical sentences. The learning task is cast as a classification prob-lem as opposed to a prediction problem; networks are taught to classify a given sequence as grammatical or ungrammatical. The two neural net-work models compared are the simple architecture proposed by Elman in [1], and the Long Short-Term Memory (LSTM) network from [4]. For comparison, a Naive Bayes (NB) classifier is also trained with bigrams to accomplish this task. Only the Elman network learns the training data better than NB, and neither network generalizes better than NB on sen-tences longer than the training sentences. The LSTM network does not generalize as well as the Elman network on a simple regular grammar, but shows comparable ability to generalize knowledge of English Gram-mar.
Conference Paper
The paper describes an automatic document classification system called is a working classification tool for natural language text, based on recurrent neural network technology. In laboratory tests, it outperformed prototypes developed with other neural network paradigms.
Conference Paper
This paper explores the capabilities of continuous time recur- rent neural networks (CTRNNs) to display reinforcement learning-like abilities on a set of T-Maze and double T-Maze navigation tasks, where the robot has to locate and "remember" the position of a reward-zone. The "learning" comes about without modifications of synapse strengths, but simply from internal network dynamics, as proposed by (12). Neural controllers are evolved in simulation and in the simple case evaluated on a real robot. The evolved controllers are analyzed and the results obtained are discussed.
Article
This paper describes how agents can learn an internal model of the world structurally by focusing on the problem of behavior-based articulation. We develop an on-line learning scheme-the so-called mixture of recurrent neural net (RNN) experts-in which a set of RNN modules become self-organized as experts on multiple levels, in order to account for the different categories of sensory-motor flow which the robot experiences. Autonomous switching of activated modules in the lower level actually represents the articulation of the sensory-motor flow. In the meantime, a set of RNNs in the higher level competes to learn the sequences of module switching in the lower level, by which articulation at a further, more abstract level can be achieved. The proposed scheme was examined through simulation experiments involving the navigation learning problem. Our dynamical system analysis clarified the mechanism of the articulation. The possible correspondence between the articulation mechanism and the attention switching mechanism in thalamo-cortical loops is also discussed.
Article
This dissertation explores the modular learning in artificial neural networks that mainly driven by the inspiration from the neurobiological basis of the human learning. The presented modularization approaches to the neural network design and learning are inspired by the engineering, complexity, psychological and neurobiological aspects. The main theme of this dissertation is to explore the organization and functioning of the brain to discover new structural and learning inspirations that can be subsequently utilized to design artificial neural network.
Article
This paper explores the use of a real-valued modular genetic algorithm to evolve continuous-time recurrent neural networks capable of sequential behavior and learning. We evolve networks that can generate a fixed sequence of outputs in response to an external trigger occurring at varying intervals of time. We also evolve networks that can learn to generate one of a set of possible sequences based upon reinforcement from the environment. Finally, we utilize concepts from dynamical systems theory to understand the operation of some of these evolved networks. A novel feature of our approach is that we assume neither an a priori discretization of states or time nor an a priori learning algorithm that explicitly modifies network parameters during learning. Rather, we merely expose dynamical neural networks to tasks that require sequential behavior and learning and allow the genetic algorithm to evolve network dynamics capable of accomplishing these tasks. 2 1. Introduction Much of the rec...
Article
Humans and other animals are exquisitely attuned to their context. Context affects almost all aspects of behavior, and it does so for the most part automatically, without conscious reasoning effort. This would be a very useful property for an artificial agent to have: upon recognizing its context, the agent's behavior would automatically adjust to fit it. This paper describes context-mediated behavior (CMB), an approach to contextsensitive behavior we have developed over the past few years for intelligent autonomous agents. In CMB, contexts are represented explicitly as contextual schemas (c-schemas). An agent recognizes its context by finding the c-schemas that match it, then it merges these to form a coherent representation of the current context. This includes not only a description of the context, but also information about how to behave in it. From that point until the next context change, knowledge for context-sensitive behavior is available with no additional effort. ...
Cognitive wheels: The frame problem in AI Minds, machines, and evolution Document classification and recurrent neural networks
  • D Dennett
Dennett, D. (1984). Cognitive wheels: The frame problem in AI. In C. Hookway (Ed.)., Minds, machines, and evolution (pp. 128–151). Cambridge University Press. Farkas, J (1995). Document classification and recurrent neural networks. In Proceedings of the 1995 conference of the centre for advanced studies on collaborative research (p. 21).
A neural reinforcement learning system. Tech. rep. TRITA-NA-P0215 Goal-seeking behavior in a connectionist model
  • C Johansson
  • Lansner
Johansson, C, & Lansner, A (2002). A neural reinforcement learning system. Tech. rep. TRITA-NA-P0215. Dept. of Numerical Analysis and Computing Science Royal Institute of Technology, Stockholm, Sweden. Portegys, T. (2001). Goal-seeking behavior in a connectionist model. Artificial Intelligence Review, 16(3), 225–253.
Maze studies with the white rat. I. Normal animals
  • Carr
Carr, H. A. (1913). Maze studies with the white rat. I. Normal animals. The Journal of Animal Behavior, 7, 259-275.
TRITA-NA-P0215. Dept. of Numerical Analysis and Computing Science Royal Institute of Technology
  • C Johansson
  • A Lansner
Johansson, C, & Lansner, A (2002). A neural reinforcement learning system. Tech. rep. TRITA-NA-P0215. Dept. of Numerical Analysis and Computing Science Royal Institute of Technology, Stockholm, Sweden.