Xavier Hinaut

National Institute for Research in Computer Science and Control | INRIA · MNEMOSYNE - Mnemonic Synergy Research Team

We present a Recurrent Neural Network (RNN) that performs thematic role assignment and can be used for Human-Robot Interaction (HRI). The RNN is trained to map sentence structures to meanings (e.g. predicates). Previously, we have shown that the model is able to generalize on English and French corpora. In this study, we investigate its ability to...

Gated working memory is defined as the capacity of holding arbitrary information at any time in order to be used at a later time. Based on electrophysiological recordings, several computational models have tackled the problem using dedicated and explicit mechanisms. We propose instead to consider an implicit mechanism based on a random recurrent ne...

Sensorimotor learning represents a challenging problem for natural and artificial systems. Several computational models have been proposed to explain the neural and cognitive mechanisms at play in the brain. In general, these models can be decomposed in three common components: a sensory system, a motor control device and a learning framework. The...

We present a simple user-friendly library called ReservoirPy based on Python scientific modules. It provides a flexible interface to implement efficient Reservoir Computing (RC) architectures with a particular focus on Echo State Networks (ESN). Advanced features of ReservoirPy allow to improve up to of computation time efficiency on a simple lapto...

Animals often demonstrate a remarkable ability to adapt to their environments during their lifetime. They do so partly due to the evolution of morphological and neural structures. These structures capture features of environments shared between generations to bias and speed up lifetime learning. In this work, we propose a computational model for st...

How does the brain represent different modes of information? Can we design a system that automatically understands what the user is thinking? Such questions can be answered by studying brain recordings like functional magnetic resonance imaging (fMRI). As a first step, the neuroscience community has contributed several large cognitive neuroscience...

Reservoir Computing (RC) is a type of recurrent neural network (RNNs) where learning is restricted to the output weights. RCs are often considered as temporal Support Vector Machines (SVMs) for the way they project inputs onto dynamic non-linear high-dimensional representations. This paradigm, mainly represented by Echo State Networks (ESNs), has b...

Working memory is the ability to maintain and manipulate information. We introduce a method based on conceptors that allows us to manipulate information stored in the dynamics (latent space) of a gated working memory model. This latter model is based on a reservoir: a random recurrent network with trainable readouts. It is trained to hold a value i...

Au commencement était la ligne, théorique et infinie, comme le temps. Cette ligne peut être vue comme une frise temporelle ou spatio-temporelle, comme une bande permettant à une « tête de lecture » de calculer ce qui est calculable. Elle peut être tracée physiquement au fur et à mesure afin de remplir l'espace de motifs improvisés ou calculés. Les...

In this article, we propose a novel architecture called hierarchical-task reservoir (HTR) suitable for real-time applications for which different levels of abstraction are available. We apply it to semantic role labeling (SRL) based on continuous speech recognition. Taking inspiration from the brain, this demonstrates the hierarchies of representat...

In learning systems, hyperparameters are parameters that are not learned but need to be set a priori. In Reservoir Computing, there are several parameters that needs to be set a priori depending on the task. Newcomers to Reservoir Computing cannot have a good intuition on which hyperparameters to tune and how to tune them. For instance, beginners o...

Domestic canaries produce complex vocal patterns embedded in various levels of abstraction. Studying such temporal organization is of particular relevance to understand how animal brains represent and process vocal inputs such as language. However, this requires a large amount of annotated data. We propose a fast and easy-to-train transducer model...

Echo States Networks (ESN) and Long-Short Term Memory networks (LSTM) are two popular architectures of Recurrent Neural Networks (RNN) to solve machine learning task involving sequential data. However, little have been done to compare their performances and their internal mechanisms on a common task. In this work, we trained ESNs and LSTMs on a Cro...

We introduce a recurrent neural network model of working memory combining short-term and long-term components. e short-term component is modelled using a gated reservoir model that is trained to hold a value from an input stream when a gate signal is on. e long-term component is modelled using conceptors in order to store inner temporal patterns (t...

We introduce a model of working memory combining short-term and long-term components. For the long-term component, we used Conceptors in order to store constant temporal patterns. For the short-term component, we used the Gated-Reservoir model: a reservoir trained to hold a triggered information from an input stream and maintain it in a readout uni...

Recently new models for language processing andlearning using Reservoir Computing have been popular. However,these models are typically not grounded in sensorimotor systemsand robots. In this paper, we develop a model of ReservoirComputing called Reservoir Parser (ResPars) for learning toparse Natural Language from grounded data coming fromhumanoid...

Gated working memory is defined as the capacity of holding arbitrary information at any time in order to be used at a later time. Based on electrophysiological recordings, several computational models have tackled the problem using dedicated and explicit mechanisms. We propose instead to consider an implicit mechanism based on a random recurrent ne...

Sensorimotor learning represents a challenging problem for artificial and natural systems. Several computational models try to explain the neural mechanisms at play in the brain to implement such learning. These models have several common components: a motor control model, a sensory system and a learning architecture. Our challenge is to build a bi...

The paper proposes a bio-inspired model for an imitative sensorimotor learning, which aims at building a map between the sensory representations of gestures (sensory targets) and their underlying motor pattern through random exploration of the motor space. An example of such learning process occurs during vocal learning in humans or birds, when you...

There has been a considerable progress these last years in speech recognition systems [13]. The word recognition error rate went down with the arrival of deep learning methods. However, if one uses cloud-based speech API and integrates it in- side a robotic architecture [33], one still encounters considerable cases of wrong sentences recognition. T...

The prefrontal cortex is known to be involved in many high-level cognitive functions, in particular, working memory. Here, we study to what extent a group of randomly connected units (namely an Echo State Network, ESN) can store and main- tain (as output) an arbitrary real value from a streamed input, i.e. can act as a sustained working memory unit...

Prefrontal cortex is known to be involved in many high-level cognitive functions, in particular working memory. Here, we study to what extent a group of randomly connected units can store and maintain (as output) an arbitrary real value from a streamed input, i.e. how such system act as a sustained working memory module without being distracted by...

The prefrontal cortex is known to be involved in many high-level cognitive functions, in particular, working memory. Here, we study to what extent a group of randomly connected units (namely an Echo State Network, ESN) can store and maintain (as output) an arbitrary real value from a streamed input, i.e. can act as a sustained working memory unit....

The understanding of how children acquire language [1][2], from phoneme to syntax, could be improved by computational models. In particular when they are integrated in robots [3]: e.g. by interacting with users [4] or grounding language cues [5]. Recently, speech recognition systems have greatly improved thanks to deep learning. However, for specif...

Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results, however computational science lags behind. In the best case, authors may provide their source code as a compressed archive and they may feel confident their research is reproducible. But this is not exactly true. Jam...

Computer science offers a large set of tools for prototyping, writing, running, testing, validating, sharing and reproducing results, however computational science lags behind. In the best case, authors may provide their source code as a compressed archive and they may feel confident their research is reproducible. But this is not exactly true. Jam...

We present a Recurrent Neural Network (RNN), namely an Echo State Network (ESN), that performs sentence comprehension and can be used for Human-Robot Interaction (HRI). The RNN is trained to map sentence structures to meanings (i.e. predicates). We have previously shown that this ESN is able to generalize to unknown sentence structures in English a...

We present a Recurrent Neural Network (RNN), namely an Echo State Network (ESN), that performs sentence comprehension and can be used for Human-Robot Interaction (HRI). The RNN is trained to map sentence structures to meanings (i.e. predicates). We have previously shown that this ESN is able to generalize to unknown sentence structures. Moreover, i...

We present a Recurrent Neural Network (RNN), namely an Echo State Network (ESN), that performs sentence comprehension and can be used for Human-Robot Interaction (HRI). The RNN is trained to map sentence structures to meanings (e.g. predicates). We have previously shown that this ESN is able to generalize to unknown sentence structures in English a...

In this paper we present a multi-modal human robot interaction architecture which is able to combine information coming from different sensory inputs, and can generate feedback for the user which helps to teach him/her implicitly how to interact with the robot. The system combines vision, speech and language with inference and feedback. The system...

To control a robot in a real-world robot scenario, a real-time parser is needed to create semantic representations from natural language which can be interpreted. The parser should be able to create the hierarchical tree-like representations without consulting external systems to show its learning capabilities. We propose an efficient Echo State Ne...

In this paper we present our experiments with an echo state network (ESN) for the task of classifying high-level human activities from video data. ESNs are recurrent neural networks which are biologically plausible, fast to train and they perform well in processing arbitrary sequential data. We focus on the integration of body motion with the infor...

How humans acquire language, and in particular two or more different languages with the same neural computing substrate, is still an open issue. To address this issue we suggest to build models that are able to process any language from the very beginning. Here we propose a developmental and neuro-inspired approach that processes sentences word by...

Video URL: http://ijcai-15.org/downloads/videos/competition/11-Humanoidly-Speaking.mp4 This video shows a friendly human-robot interaction using humanoid Nao robots. The speaker teaches the robot some names of objects using speech. This work shows the successful integration of three different projects mainly using Artificial Neural Networks: (1) o...

In previous research a model for thematic role assignment (θRARes) was pro-posed, using the Reservoir Computing paradigm. This language comprehension model consisted of a recurrent neural network (RNN) with fixed random con-nections which models distributed processing in the prefrontal cortex, and an output layer which models the striatum. In contr...

One of the principal functions of human language is to allow people to coordinate joint action. This includes the description of events, requests for action, and their organization in time. A crucial component of language acquisition is learning the grammatical structures that allow the expression of such complex meaning related to physical events....

In order to be able to understand a conversation in interaction, a robot, has to first understand the language used by his interlocutor. A central aspect of language learning is adaptability. Individuals can learn new words and new grammatical structures. We have developed learning methods that allow the humanoid robot iCub to robot can learn new l...

Sentence processing takes place in real-time. Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. Recent neurophysiological studies in humans suggest that the fronto-striatal system (frontal cortex, and striatum - the major input locus of the basal ganglia) plays a crucial ro...

Simulation results with same conditions as Experiment 1 but with reservoir size N = 100, and activation time AT = 1. Note that when compared with Figures S1 and S2, the temporal profile of activation for the output neurons is globally the same, but with increased variability. (TIF)

Simulation results with same conditions as Experiment 1 but with reservoir size N = 1000, and activation time AT = 20. Note that for each output neuron, the temporal profile of activation is the same as that in Figure 2, obtained with N = 300, AT = 20. (TIF)

Simulation results with same conditions as Experiment 1 but with reservoir size N = 100, and activation time AT = 20. Note that when compared with Figure S1, the temporal profile of activation for the output neurons is globally the same, but with increased variability. (TIF)

Read-me file with instructions on how to install and run the model given in the Zipped Archive S1. (TXT)

Detailed neural activity for all constructions tested in Experiment 1. The format of this data is identical to that of Figures 2–7 of the main text. (PDF)

Additional information about the set of grammatical constructions for Experiments 1–4, and tests with different input activation times. (DOC)

This is a zipped file containing python code using the Oger toolbox for running simulations corresponding to Experiments 1–4 of our PLoS ONE 2013 (Hinaut & Dominey), along with documentation for installation and running the scripts.

The 462 construction corpus. (RTF)

Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. The current research provides a possible explanation of how certain aspects of this on-line language processing can occur, based on the dynamics of recurrent cortical networks. We simulate prefrontal area BA47 as a recurrent...

The goal of this research is to provide a real-time and adaptive spoken langue interface between humans and a humanoid robot. The system should be able to learn new grammatical constructions in real-time, and then use them immediately following or in a later interactive session. In order to achieve this we use a recurrent neural network of 500 neur...

Categorical encoding is crucial for mastering large bodies of related sensory-motor experiences, but what is its neural substrate? In an effort to respond to this question, recent single-unit recording studies in the macaque lateral prefrontal cortex (LPFC) have demonstrated two characteristic forms of neural encoding of the sequential structure of...

Categorical encoding is crucial for mastering large bodies of related sensory experiences. Recent single-unit recording studies in the macaque prefrontal cortex have demonstrated two characteristic forms of neural encoding of the sequential structure of the animal's behaviour. One population of neurons encodes the specific behavioural sequences. A...