Pablo Barros

Hamburg University | UHH · Department of Informatics

Recent advances in reinforcement learning with social agents have allowed such models to achieve human-level performance on certain interaction tasks. However, most interactive scenarios do not have performance alone as an end-goal; instead, the social impact of these agents when interacting with humans is as important and largely unexplored. In th...

Human-robot interaction (HRI) benefits greatly from advances in the machine learning field as it allows researchers to employ high-performance models for perceptual tasks like detection and recognition. Especially deep learning models, either pre-trained for feature extraction or used for classification, are now established methods to characterize...

Recent advances in reinforcement learning with social agents have allowed such models to achieve human-level performance on specific interaction tasks. However, most interactive scenarios do not have a version alone as an end goal; instead, the social impact of these agents when interacting with humans is as important and largely unexplored. In thi...

One of the most common health risks for senior citizens is a falling event, and to reduces the risk of death, a fall needs to be quickly reported. Thus, automatic fall detection systems were proposed to mitigate the falling problem, most of them relying on image detection. However, identifying a fall from a sequence of images is challenging because...

Current facial expression recognition systems demand an expensive re-training routine when deployed to different scenarios than they were trained for. Biasing them towards learning specific facial characteristics, instead of performing typical transfer learning methods, might help these systems to maintain high performance in different tasks, but w...

In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries’ goals at the same time. Besides dealing with the increased dynamics of the scenarios due to the opponents’ actions, they usually have to understand how to overcome the opponent’s strategies. Most of the common solutions,...

Collaborative interactions require social robots to share the users’ perspective on the interactions and adapt to the dynamics of their affective behaviour. Yet, current approaches for affective behaviour generation in robots focus on instantaneous perception to generate a one-to-one mapping between observed human expressions and static robot actio...

Facial expression recognition, as part of an affective computing system, usually relies on solid performance metrics to be successful. These metrics depend significantly on the affective context in which one evaluates this system. While presenting excellent performance on the dataset it was trained on, a facial expression recognition model might dr...

Recent studies have revealed the key importance of modelling personality in robots to improve interaction quality by empowering them with social-intelligence capabilities. Most research relies on verbal and non-verbal features related to personality traits that are highly context-dependent. Hence, analysing how humans behave in a given context is c...

Reinforcement learning simulation environments pose an important experimental test bed and facilitate data collection for developing AI-based robot applications. Most of them, however, focus on single-agent tasks, which limits their application to the development of social agents. This study proposes the Chef’s Hat simulation environment, which imp...

Automatic systems to monitor people and subsequently improve people’s lives have been emerging in the last few years, and currently, they are capable of identifying many activities of daily living (ADLs). An important field of research in this context is the monitoring of health risks and the identification of falls. It is estimated that every year...

The current COVID-19 pandemic has shown us that we are still facing unpredictable challenges in our society. The necessary constrain on social interactions affected heavily how we envision and prepare the future of social robots and artificial agents in general. Adapting current affective perception models towards constrained perception based on th...

In domains where computational resources and labeled data are limited, such as in robotics, deep networks with millions of weights might not be the optimal solution. In this paper, we introduce a connectivity scheme for pyramidal architectures to increase their capacity for learning features. Experiments on facial expression recognition of unseen p...

This paper describes the design of an interactive game between humans and a robot that makes it possible to observe, analyze, and model competitive strategies and affective interactions with the aim to dynamically generate appropriate responses or initiations of a robot. We apply an iterative design process that applied several pilot evaluations to...

Social robots able to continually learn facial expressions could progressively improve their emotion recognition capability towards people interacting with them. Semi-supervised learning through ensemble predictions is an efficient strategy to leverage the high exposure of unlabelled facial expressions during human-robot interactions. Traditional e...

Effectively recognising and applying emotions to interactions is a highly desirable trait for social robots. Implicitly understanding how subjects experience different kinds of actions and objects in the world is crucial for natural HRI interactions, with the possibility to perform positive actions and avoid negative actions. In this paper, we util...

A computational system able to automatically and efficiently detect and classify falls would be beneficial for monitoring the elderly population and speed up the assistance proceedings, reducing the risk of prolonged injuries and death. One of the most common problems in such systems is the high number of false-positives in their recognition scheme...

Effective tutoring during motor learning requires to provide the appropriate physical assistance to the learners, but at the same time to assess and adapt to their state, to avoid frustration. With the aim of endowing robot tutors with these abilities, we designed an experiment in which participants had to acquire a new motor ability - balancing an...

Recent advances in reinforcement learning with social agents have allowed us to achieve human-level performance on some interaction tasks. However, most interactive scenarios do not have as end-goal performance alone; instead, the social impact of these agents when interacting with humans is as important and, in most cases, never explored properly....

Current state-of-the-art models for automatic facial expression recognition (FER) are based on very deep neural networks that are effective but rather expensive to train. Given the dynamic conditions of FER, this characteristic hinders such models of been used as a general affect recognition. In this paper, we address this problem by formalizing th...

Cover Page, Preface, Invited Speakers, Organizing Committee, Venue, Sponsors, Organizers, and Awards.

Collaborative interactions require social robots to adapt to the dynamics of human affective behaviour. Yet, current approaches for affective behaviour generation in robots focus on instantaneous perception to generate a one-to-one mapping between observed human expressions and static robot actions. In this paper, we propose a novel framework for p...

Predicting affective information from human faces became a popular task for most of the machine learning community in the past years. The development of immense and dense deep neural networks was backed by the availability of numerous labeled datasets. These models, most of the time, present state-of-the-art results in such benchmarks, but are very...

Current state-of-the-art models for automatic Facial Expression Recognition (FER) are based on very deep neural networks that are effective but rather expensive to train. Given the dynamic conditions of FER, this characteristic hinders such models of been used as a general affect recognition. In this paper, we address this problem by formalizing th...

Designing the decision-making processes of artificial agents that are involved in competitive interactions is a challenging task. In a competitive scenario, the agent does not only have a dynamic environment but also is directly affected by the opponents' actions. Observing the Q-values of the agent is usually a way of explaining its behavior, howe...

Recently deep generative models have achieved impressive results in the field of automated facial expression editing. However, the approaches presented so far presume a discrete representation of human emotions and are therefore limited in the modelling of non-discrete emotional expressions. To overcome this limitation, we explore how continuous em...

Recent models of emotion recognition strongly rely on supervised deep learning solutions for the distinction of general emotion expressions. However , they are not reliable when recognizing on-line and personalized facial expressions, e.g., for person-specific affective understanding. In this paper, we present a neural model based on a conditional...

Current state-of-the-art models for automatic FER are based on very deep neural networks that are difficult to train. This makes it challenging to adapt these models to changing conditions, a requirement from FER models given the subjective nature of affect perception and understanding. In this paper, we address this problem by formalizing the Face...

Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments a...

In contrast to many established emotion recognition systems, convolutional neural networks do not rely on handcrafted features to categorize emotions. Although achieving state-of-the-art performances, it is still not fully understood what these networks learn and how the learned representations correlate with the emotional characteristics of speech...

Expectation learning is a unsupervised learning process which uses multisensory bindings to enhance unisensory perception. For instance, as humans, we learn to associate a barking sound with the visual appearance of a dog, and we continuously fine-tune this association over time, as we learn, e.g., to associate high-pitched barking with small dogs....

Facial recognition tasks like identity, age, gender, and emotion recognition received substantial attention in recent years. Their deployment in robotic platforms became necessary for the characterization of most of the non-verbal Human-Robot Interaction (HRI) scenarios. In this regard, deep convolution neural networks have shown to be effective on...

Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments a...

Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments a...

Abstract—Processing human affective behavior is important for developing intelligent agents that interact with humans in complex interaction scenarios. A large number of current approaches that address this problem focus on classifying emotion expressions by grouping them into known categories. Such strategies neglect, among other aspects, the impa...

Processing human affective behavior is important for developing intelligent agents that interact with humans in complex interaction scenarios. A large number of current approaches that address this problem focus on classifying emotion expressions by grouping them into known categories. Such strategies neglect, among other aspects, the impact of the...

Convolutional Neural Networks (CNNs) have shown promising results for various computer vision tasks. Despite their success, localizing the ball in real-world RoboCup Soccer scenes is still challenging. Especially considering real-time requirements and the limited computing power of humanoid robots. Another important reason is the lack of training a...

Abstract— Recently deep generative models have achieved impressive results in the field of automated facial expression editing. However, the approaches presented so far presume a discrete representation of human emotions and are therefore limited in the modelling of non-discrete emotional expressions.To overcome this limitation, we explore how cont...

Recent models of emotion recognition strongly rely on supervised deep learning solutions for the distinction of general emotion expressions. However, they are not reliable when recognizing online and personalized facial expressions, e.g., for person-specific affective understanding. In this paper, we present a neural model based on a conditional ad...

The efficient integration of multisensory observations is a key property of the brain that yields the robust interaction with the environment. However, artificial multisensory perception remains an open issue especially in situations of sensory uncertainty and conflicts. In this work, we extend previous studies on audio-visual (AV) conflict resolut...

A moving robot or moving camera causes motion blur in the robot’s vision and distorts recorded images. We show that motion blur, differing lighting, and other distortions heavily affect the object localization performance of deep learning architectures for RoboCup Humanoid Soccer scenes. The paper proposes deep conditional generative models to appl...

Abstract— Social robots able to continually learn facial expressions could progressively improve their emotion recognition capability towards people interacting with them. Semi-supervised learning through ensemble predictions is an efficient strategy to leverage the high exposure of unlabelled facial expressions during human-robot interactions. Tra...

Crossmodal conflict resolution is crucial for robot sensorimotor coupling through the interaction with the environment, yielding swift and robust behaviour also in noisy conditions. In this paper, we propose a neurorobotic experiment in which an iCub robot exhibits human-like responses in a complex crossmodal environment. To better understand how h...

Emotional concepts play a huge role in our daily life since they take part into many cognitive processes: from the perception of the environment around us to different learning processes and natural communication. Social robots need to communicate with humans, which increased also the popularity of affective embodied models that adopt different emo...

Our daily perceptual experience is driven by different neural mechanisms that yield multisensory interaction as the interplay between exogenous stimuli and endogenous expectations. While the interaction of multisensory cues according to their spatiotemporal properties and the formation of multisensory feature-based representations have been widely...

Being able to recognize emotions in human users is considered a highly desirable trait in Human-Robot Interaction (HRI) scenarios. However, most contemporary approaches rarely attempt to apply recognized emotional features in an active manner to modulate robot decision-making and dialogue for the benefit of the user. In this position paper, we prop...

Being able to recognize emotions in human users is considered a highly desirable trait in Human-Robot Interaction (HRI) scenarios. However, most contemporary approaches rarely attempt to apply recognized emotional features in an active manner to modulate robot decision-making and dialogue for the benefit of the user. In this position paper, we prop...

Human-Robot Interaction (HRI) studies, particularly the ones designed around social robots, use emotions as important building blocks for interaction design. In order to provide a natural interaction experience, these social robots need to recognise the emotions expressed by the users across various modalities of communication and use them to estim...

Abstract—The human brain is able to learn, generalize, and predict crossmodal stimuli which help us to understand the world around us. Some characteristics of crossmodal learning inspired some computational models but most of the solutions only go as far as to implement strategies for early or late crossmodal fusion. In this paper, we propose the u...

This paper is the basis paper for the accepted IJCNN challenge One-Minute Gradual-Emotion Recognition (OMG-Emotion) by which we hope to foster long-emotion classification using neural models for the benefit of the IJCNN community. The proposed corpus has as the novelty the data collection and annotation strategy based on emotion expressions which e...

In domains where computational resources and labeled data are limited, such as in robotics, deep networks with millions of weights might not be the optimal solution. In this paper, we introduce a con-nectivity scheme for pyramidal architectures to increase their capacity for learning features. Experiments on facial expression recognition of unseen...

This paper is the basis paper for the accepted IJCNN challenge One-Minute Gradual-Emotion Recognition (OMG-Emotion) by which we hope to foster long-emotion classification using neural models for the benefit of the IJCNN community. The proposed corpus has as the novelty the data collection and annotation strategy based on emotion expressions which e...

Crossmodal conflict resolution is a crucial component of robot sensorimotor coupling through interaction with the environment for swift and robust behaviour also in noisy conditions. In this paper, we propose a neurorobotic experiment in which an iCub robot exhibits human-like responses in a complex crossmodal environment. To better understand how...

The human brain is able to learn, generalize, and predict crossmodal stimuli. Learning by expectation fine-tunes crossmodal processing at different levels, thus enhancing our power of generalization and adaptation in highly dynamic environments. In this paper, we propose a deep neural architecture trained by using expectation learning accounting fo...

In RoboCup soccer, ball localization is an important and challenging task, especially since the last change of the rule which allows 50% of the ball’s surface to be of any color or pattern while the rest must remain white. Multi-color balls have changing color histograms and patterns in dependence of the current orientation and movement. This paper...

The recognition of emotions plays an important role in our daily life and is essential for social communication. Although multiple studies have shown that body expressions can strongly convey emotional states, emotion recognition from body motion patterns has received less attention than the use of facial expressions. In this paper, we propose a se...

Th