Pablo Lanillos

Pablo Lanillos
Radboud University | RU · Donders Institute for Brain, Cognition, and Behaviour

PhD
Coordinating Spikeference and DeepSelf projects

About

72
Publications
13,170
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
695
Citations
Introduction
I am an Assistant Professor in Cognitive AI researching neuroscience-inspired artificial intelligence and machine learning approaches for perception and action. Focusing on variational learning, probabilistic deep learning, predictive coding and active inference. My goal is to develop algorithms that allow robots to perceive and act with their body as humans do and at the same time disentangle how animals construct their self-representation through sensorimotor learning.
Additional affiliations
November 2019 - present
Radboud University
Position
  • Professor (Assistant)
Description
  • Tenured Associate Professor and Associate Principal Investigator
May 2017 - November 2019
Technische Universität München
Position
  • Marie Skłodowska Curie
October 2015 - April 2017
Technische Universität München
Position
  • PostDoc Position

Publications

Publications (72)
Article
Full-text available
One of the biggest challenges in robotics is interacting under uncertainty. Unlike robots, humans learn, adapt and perceive their body as a unity when interacting with the world. Here we investigate the suitability of Active inference, a computational model proposed for the brain and governed by the free-energy principle, for robotic body perceptio...
Article
Full-text available
The perception of our body in space is flexible and manipulable. The predictive brain hypothesis explains this malleability as a consequence of the interplay between incoming sensory information and our body expectations. However, given the interaction between perception and action, we might also expect that actions would arise due to prediction er...
Preprint
Full-text available
Active inference is a mathematical framework which originated in computational neuroscience as a theory of how the brain implements action, perception and learning. Recently, it has been shown to be a promising approach to the problems of state-estimation and control under uncertainty, as well as a foundation for the construction of goal-driven beh...
Article
The field of motor control has long focused on the achievement of external goals through action (e.g., reaching and grasping objects). However, recent studies in conditions of multisensory conflict, such as when a subject experiences the rubber hand illusion or embodies an avatar in virtual reality, reveal the presence of unconscious movements that...
Preprint
Full-text available
Computational models of visual attention in artificial intelligence and robotics have been inspired by the concept of a saliency map. These models account for the mutual information between the (current) visual information and its estimated causes. However, they fail to consider the circular causality between perception and action. In other words,...
Article
Full-text available
Recent advances in neuroscience have characterised brain function using mathematical formalisms and first principles that may be usefully applied elsewhere. In this paper, we explain how active inference—a well-known description of sentient behaviour from neuroscience—can be exploited in robotics. In short, active inference leverages the processes...
Preprint
The field of motor control has long focused on the achievement of external goals through action (e.g., reaching and grasping objects). However, recent studies in conditions of multisensory conflict, such as when a subject experiences the rubber hand illusion or embodies an avatar in virtual reality, reveal the presence of unconscious movements that...
Preprint
Adaptation to external and internal changes is major for robotic systems in uncertain environments. Here we present a novel multisensory active inference torque controller for industrial arms that shows how prediction can be used to resolve adaptation. Our controller, inspired by the predictive brain hypothesis, improves the capabilities of current...
Preprint
Spiking neural networks are a promising approach towards next-generation models of the brain in computational neuroscience. Moreover, compared to classic artificial neural networks, they could serve as an energy-efficient deployment of AI by enabling fast computation in specialized neuromorphic hardware. However, training deep spiking neural networ...
Preprint
Knowing the position of the robot in the world is crucial for navigation. Nowadays, Bayesian filters, such as Kalman and particle-based, are standard approaches in mobile robotics. Recently, end-to-end learning has allowed for scaling-up to high-dimensional inputs and improved generalization. However, there are still limitations to providing reliab...
Preprint
Despite the potential of active inference for visual-based control, learning the model and the preferences (priors) while interacting with the environment is challenging. Here, we study the performance of a deep active inference (dAIF) agent on OpenAI's car racing benchmark, where there is no access to the car's state. The agent learns to encode th...
Preprint
Unlike robots, humans learn, adapt and perceive their bodies by interacting with the world. Discovering how the brain represents the body and generates actions is of major importance for robotics and artificial intelligence. Here we discuss how neuroscience findings open up opportunities to improve current estimation and control algorithms in robot...
Preprint
Active inference, a theoretical construct inspired by brain processing, is a promising alternative to control artificial agents. However, current methods do not yet scale to high-dimensional inputs in continuous control. Here we present a novel active inference torque controller for industrial arms that maintains the adaptive characteristics of pre...
Article
Self-recognition or self-awareness is a capacity attributed typically only to humans and few other species. The definitions of these concepts vary and little is known about the mechanisms behind them. However, there is a Turing test-like benchmark: the mirror self-recognition, which consists in covertly putting a mark on the face of the tested subj...
Chapter
Full-text available
Deep active inference has been proposed as a scalable approach to perception and action that deals with large policy and state spaces. However, current models are limited to fully observable domains. In this paper, we describe a deep active inference model that can learn successful policies directly from high-dimensional sensory inputs. The deep le...
Chapter
Full-text available
Understanding how perception and action deal with sensorimotor conflicts, such as the rubber-hand illusion (RHI), is essential to understand how the body adapts to uncertain situations. Recent results in humans have shown that the RHI not only produces a change in the perceived arm location, but also causes involuntary forces. Here, we describe a d...
Preprint
Self-recognition or self-awareness is a capacity attributed typically only to humans and few other species. The definitions of these concepts vary and little is known about the mechanisms behind them. However, there is a Turing test-like benchmark: the mirror self-recognition, which consists in covertly putting a mark on the face of the tested subj...
Preprint
Deep active inference has been proposed as a scalable approach to perception and action that deals with large policy and state spaces. However, current models are limited to fully observable domains. In this paper, we describe a deep active inference model that can learn successful policies directly from high-dimensional sensory inputs. The deep le...
Preprint
Understanding how perception and action deal with sensorimotor conflicts, such as the rubber-hand illusion (RHI), is essential to understand how the body adapts to uncertain situations. Recent results in humans have shown that the RHI not only produces a change in the perceived arm location, but also causes involuntary forces. Here, we describe a d...
Preprint
Full-text available
The perception of our body in space is flexible and manipulable. The predictive brain hypothesis explains this malleability as a consequence of the interplay between incoming sensory information and our body expectations. However, given the interaction between perception and action, we might also expect that actions would arise due to prediction er...
Conference Paper
Full-text available
Self/other distinction and self-recognition are important skills for interacting with the world, as it allows humans to differentiate own actions from others and be self-aware. However, only a selected group of animals, mainly high order mammals such as humans, have passed the mirror test, a behavioural experiment proposed to assess self-recognitio...
Preprint
Self/other distinction and self-recognition are important skills for interacting with the world, as it allows humans to differentiate own actions from others and be self-aware. However, only a selected group of animals, mainly high order mammals such as humans, has passed the mirror test, a behavioural experiment proposed to assess self-recognition...
Article
Full-text available
We present an active visual search model for finding objects in unknown environments. The proposed algorithm guides the robot towards the sought object using the relevant stimuli provided by the visual sensors. Existing search strategies are either purely reactive or use simplified sensor models that do not exploit all the visual information availa...
Preprint
We present a pixel-based deep Active Inference algorithm (PixelAI) inspired in human body perception and successfully validated in robot body perception and action as a use case. Our algorithm combines the free energy principle from neuroscience, rooted in variational inference, with deep convolutional decoders to scale the algorithm to directly de...
Article
Full-text available
This survey presents the most relevant neural network models of autism spectrum disorder and schizophrenia, from the first connectionist models to recent deep neural network architectures. We analyzed and compared the most representative symptoms with its neural model counterpart, detailing the alteration introduced in the network that generates ea...
Conference Paper
Full-text available
Patients who lost their ability to move and talk are often socially deprived. To assist them, we present a prototype of a humanoid robotic system that aims to extend the social sphere and autonomy of the patients via an EEG based brain-computer interface. The system enables multi-modal and bidirectional communication. It empowers the patient to int...
Preprint
Full-text available
Perceptual hallucinations are present in neurological and psychiatric disorders and amputees. While the hallucinations can be drug-induced, it has been described that they can even be provoked in healthy subjects. Understanding their manifestation could thus unveil how the brain processes sensory information and might evidence the generative nature...
Preprint
This survey presents the most relevant neural network models of autism spectrum disorder and schizophrenia, from the first connectionist models to recent deep network architectures. We analyzed and compared the most representative symptoms with its neural model counterpart, detailing the alteration introduced in the network that generates each of t...
Preprint
Full-text available
One of the biggest challenges in robotics systems is interacting under uncertainty. Unlike robots, humans learn, adapt and perceive their body as a unity when interacting with the world. We hypothesize that the nervous system counteracts sensor and motor uncertainties by unconscious processes that robustly fuse the available information for approxi...
Preprint
Artificial self-perception is the machine ability to perceive its own body, i.e., the mastery of modal and intermodal contingencies of performing an action with a specific sensors/actuators body configuration. In other words, the spatio-temporal patterns that relate its sensors (e.g. visual, proprioceptive, tactile, etc.), its actions and its body...
Preprint
We present an active visual search model for finding objects in unknown environments. The proposed algorithm guides the robot towards the sought object using the relevant stimuli provided by the visual sensors. Existing search strategies are either purely reactive or use simplified sensor models that do not exploit all the visual information availa...
Preprint
Full-text available
Humans can experience fake body parts as theirs just by simple visuo-tactile synchronous stimulation. This body-illusion is accompanied by a drift in the perception of the real limb towards the fake limb, suggesting an update of body estimation resulting from stimulation. This work compares body limb drifting patterns of human participants, in a ru...
Preprint
Full-text available
The predictive functions that permit humans to infer their body state by sensorimotor integration are critical to perform safe interaction in complex environments. These functions are adaptive and robust to non-linear actuators and noisy sensory information. This paper introduces a computational perceptual model based on predictive processing that...
Data
Video to A Tactile-Based Framework for Active Object Learning and Discrimination using Multimodal Robotic Skin
Conference Paper
Full-text available
In this paper we discuss the enactive self from a computational point of view and study the suitability of current methods to instantiate it onto robots. As an assumption, we consider any cognitive agent as an autonomous system that constructs its identity by continuous interaction with the environment. We start examining algorithms to learn the bo...
Article
Full-text available
In this paper, we propose a complete probabilistic tactile-based framework to enable robots to autonomously explore unknown workspaces and recognize objects based on their physical properties. Our framework consists of three components: (1) an active pre-touch strategy to efficiently explore unknown workspaces; (2) an active touch learning method t...
Article
In this text, we will present a probabilistic solution for robust gaze estimation in the context of human-robot interaction. Gaze estimation, in the sense of continuously assessing gaze direction of an interlocutor so as to determine his/her focus of visual attention, is important in several important computer vision applications, such as the devel...
Article
Full-text available
We address self-perception in robots as the key for world understanding and causality interpretation. We present a self-perception mechanism that enables a humanoid robot to understand certain sensory changes caused by naive actions during interaction with objects. Visual, proprioceptive and tactile cues are combined via artificial attention and pr...
Conference Paper
The development of breakthrough technologies helps the deployment of robotic systems in the industry. The implementation and integration of such technologies will improve productivity, flexibility and competitiveness, in diverse industrial settings specially for small and medium enterprises. In this paper we present a framework that integrates thre...
Poster
Inducing simple causality using visual, proprioceptive and tactile cues correlation and artificial attention during interaction.
Conference Paper
We address self-perception and object discovery by integrating multimodal tactile, proprioceptive and visual cues. Considering sensory signals as the only way to obtain relevant information about the environment, we enable a humanoid robot to infer potential usable objects relating visual self-detection with tactile cues. Hierarchical Bayesian mode...
Conference Paper
Full-text available
Human gaze is one of the most important cue for social robotics due to its embedded intention information. Discovering the location or the object that an interlocutor is staring at, gives the machine some insight to perform the correct attentional behaviour. This work presents a fast voxel traversal algorithm for estimating the potential locations...
Conference Paper
Full-text available
In this text, we present the principles that allow the tractable implementation of exact inference processes concerning a group of widespread classes of Bayesian generative models, which have until recently been deemed as intractable whenever formulated using high-dimensional joint distributions. We will demonstrate the usefulness of such a princip...
Conference Paper
Full-text available
In this paper we present proof-of-concept for a novel solution consisting of a short-term 3D memory for artificial attention systems, loosely inspired in perceptual processes believed to be implemented in the human brain. Our solution supports the implementation of multisen-sory perception and stimulus-driven processes of attention. For this purpos...
Article
Full-text available
The minimum time search in uncertain domains is a searching task, which appears in real world problems such as natural disasters and sea rescue operations, where a target has to be found, as soon as possible, by a set of sensor-equipped searchers. The automation of this task, where the time to detect the target is critical, can be achieved by new p...
Conference Paper
Full-text available
The human ability of unconsciously attending to social signals , together with other even more primitive automatic at-tentional processes, has been argued in the literature to play an important part in social interaction. In this paper, we will argue that the evaluation of the influence of these unconscious perceptual processes in social interactio...
Thesis
Full-text available
This thesis is concerned with the development of an autonomous system to search a dynamic target in the minimum possible time in uncertain environments. The minimum time search problem consists in determining the best sequence of actions (observations) to find a target (object) with uncertain location in the minimum time possible. In more colloquia...
Article
Full-text available
This paper proposes a Bayesian approach for minimizing the time of finding an object of uncertain location and dynamics using several moving sensing agents with constrained dynamics. The approach exploits twice the Bayesian theory: on one hand, it uses a Bayesian formulation of the objective functions that compare the constrained paths of the agent...
Article
Full-text available
This paper presents a state of the art of the estimation algorithms dealing with Out-of-Sequence (OOS) measurements for non-linearly modeled systems. The state of the art includes a critical analysis of the algorithm properties that takes into account the applicability of these techniques to autonomous mobile robot navigation based on the fusion of...
Article
Full-text available
This paper formulates and proposes a discrete solution for the problem of finding a lost target under uncertainty in minimum time (Minimum Time Search). Given a searching region where some information about the target is known but uncertain (i.e. location and dynamics), and a searching agent with constrained dynamics, we provide two decision making...
Conference Paper
This work presents a strategy to track and describe the boundary of an environmental surface using a sequence of incomplete boundary images coming from an UAV with a camera onboard. The algorithm uses all captured images to feed the trajectory, which drives the UAV towards the next point to take the next shot, until the identification is completed....
Article
Full-text available
La generación de trayectorias y la replanificación de las mismas en entornos hostiles para UAVs (Unmanned Aerial Vehicles) es una disciplina en auge. Los entornos hostiles se caracterizan por la presencia de amenazas, modeladas aquí como radares. Inicialmente se planifica una ruta. Si en vuelo surgen nuevas amenazas, la ruta inicial se replanifica....