Luis Merino

• PhD
• Professor (Associate) at Pablo de Olavide University

This paper presents a novel approach to efficiently parameterize and estimate the state of a hanging tether for path and trajectory planning of a UGV tied to a UAV in a marsupial configuration. Most implementations in the state of the art assume a taut tether or make use of the catenary curve to model the shape of the hanging tether. The catenary m...

4D millimeter-wave (mmWave) radars are sensors that provide robustness against adverse weather conditions (rain, snow, fog, etc.), and as such they are increasingly being used for odometry and SLAM applications. However, the noisy and sparse nature of the returned scan data proves to be a challenging obstacle for existing point cloud matching based...

This paper presents a ROS 2-based simulator framework for tethered UAV-UGV marsupial systems in Gazebo. The framework models interactions among a UAV, a UGV, and a winch with dynamically adjustable length and slack of the tether. It supports both manual control and automated trajectory tracking, with the winch adjusting the length of the tether bas...

Non-verbal communication, especially eye contact, plays a crucial role in human interaction. Integrating eye contact capabilities into robotic behavior presents challenges that require comprehensive interdisciplinary collaboration from the fields of vision science and robotics. This study introduces an innovative methodology to improve the user's p...

The development and integration of robots capable of expressing gaze directionality through eye-head movements are crucial for effective human-robot interaction, especially for those with eye designs on 2D screens. Our proposed mutual eye-head gaze model aligns eye movements with head/body rotation, incorporating an attention engine for estimating...

In this paper, we design a hierarchical set of socio-emotional features to understand the effect of cross-cultural mediation facilitated by a social robot between two remote groups of schoolchildren in Japan and in Australia. We also equip the robot with behaviors that maximize children's participation and stimulate socio-emotional interaction whic...

This work presents the Human Navigation Simulator ( HuNavSim ), a novel open-source tool for the simulation of different human-agent navigation behaviors in scenarios with mobile robots. The tool, the first programmed under the ROS 2 framework, can be used together with different well-known robotics simulators like Gazebo. The main goal is to faci...

This letter addresses the problem of trajectory planning in a marsupial robotic system consisting of an unmanned aerial vehicle (UAV) linked to an unmanned ground vehicle (UGV) through a non-taut tether with controllable length. To the best of our knowledge, this is the first method that addresses the trajectory planning of a marsupial UGV-UAV with...

This work presents a framework for a visual- auditory attention-driven robot eye-head gaze movement, which combines visual and auditory inputs to determine the direction of movement for a social robot. The framework computes the most salient changes in position by considering both visual and auditory cues. The proposed system was implemented on Har...

Mobile robots require knowledge of the environment, especially of humans located in its vicinity. While the most common approaches for detecting humans involve computer vision, an often overlooked hardware feature of robots for people detection are their 2D range finders. These were originally intended for obstacle avoidance and mapping/SLAM tasks....

This work presents the Human Navigation Simulator (HuNavSim), a novel open-source tool for the simulation of different human-agent navigation behaviors in scenarios with mobile robots. The tool, the first programmed under the ROS 2 framework, can be employed along with different well- known robotics simulators like Gazebo. The main goal is to ease...

This letter addresses the problem of trajectory planning in a marsupial robotic system consisting of an unmanned aerial vehicle (UAV) linked to an unmanned ground vehicle (UGV) through a non-taut tether withcontrollable length. To the best of our knowledge, this is the first method that addresses the trajectory planning of a marsupial UGV-UAV with...

This article presents a people perception software architecture and its implementation, focused on the information of interest from the point of view of a social robot. The key modules employed to get the different people features, such as the body parts location, the face and hands information, and the speech, from a set of possible devices and co...

Optical range sensors such as LiDAR and range cameras have become the most common devices for robot localization and navigation tasks. However, their performance can be degraded by meteorological hazards, such as fog, smoke, or rain. This paper proposes a new method to combine information from LiDAR sensors and low-cost RADAR sensors in structured...

This paper addresses the problem of trajectory planning in a marsupial robotic system consisting of an unmanned aerial vehicle (UAV) linked to an unmanned ground vehicle (UGV) through a non-taut tether that has a controllable length. The objective is to determine a synchronized collision-free trajectory for the three marsupial system agents: UAV, U...

This paper presents a framework for urban firefighting with a heterogeneous aerial/ground robot team. The system was developed to address Challenge 3 of the MBZIRC’20. The challenge required autonomously detecting, locating, and extinguishing fires on multiple interior floors and on exterior facade and ground surfaces. Our multi-robot system consis...

In recent years, high-speed navigation and environment interaction in the context of aerial robotics has become a field of interest for several academic and industrial research studies. In particular, Search and Intercept (SaI) applications for aerial robots pose a compelling research area due to their potential usability in several environments. N...

En el presente trabajo se propone un marco de referencia para el aprendizaje de interacciones entre personas y robots, basado en el uso conjunto de una técnica de aprendizaje sin supervisión y de un planificador de muestreo de configuraciones. Particularmente, se hace uso de los Modelos Mixtos Gausianos (GMMs en inglés) para modelar la interacción...

Este trabajo presenta un algoritmo para el aprendizaje de comportamientos de navegación a partir de demostraciones usando árboles de exploración aleatoria óptimos (RRT*) como planificador de caminos. El algoritmo de aprendizaje combina las técnicas de Inverse Reinforcement Learning (IRL) y RRT* para aprender los pesos de la función de coste a parti...

This paper summarizes the latest advances of the EU Project SIAR-ECHORD++, whose main objective is the design of a new robotic platform for inspecting visitable sewers. The SIAR robot aims to determine the sewer serviceability, to identify critical structural defects, to perform sewer monitoring and eventually to take water or gas samples of the en...

Scene graph generation from images is a task of great interest to applications such as robotics, because graphs are the main way to represent knowledge about the world and regulate human-robot interactions in tasks such as Visual Question Answering (VQA). Unfortunately, its corresponding area of machine learning is still relatively in its infancy,...

Scene graph generation from images is a task of great interest to applications such as robotics, because graphs are the main way to represent knowledge about the world and regulate human-robot interactions in tasks such as Visual Question Answering (VQA). Unfortunately, its corresponding area of machine learning is still relatively in its infancy,...

In recent years, high-speed navigation and environment interaction in the context of aerial robotics has become a field of interest for several academic and industrial research studies. In particular, Search and Intercept (SaI) applications for aerial robots pose a compelling research area due to their potential usability in several environments. N...

This paper describes the development of a reactive behavioral response framework for the tabletop robot Haru. The framework enables the robot to react to external stimuli through a repertoire of expressive routines. The behavioral response framework is inspired by the simple reactive behaviors of organisms (e.g. reflexes) based on a bottom-up atten...

The experimental tabletop robot Haru, used for affective telepresence research, enables a teleoperator to communicate a variety of information to a remote user through the robotic medium from a distance. However, the robot’s rich communicative modality poses some problems to the teleoperator. Based on their experience of controlling the robot, tele...

In this paper, we explore the storytelling potential of a robot. We exploit the use of creative contents that maximize the embodied communication affordance of the empathic robot Haru. We identify the elements in storytelling such as narration, agency, engagement and education and synthesized these into the robot. Through effective design we invest...

This paper presents a human–robot co-working system to be applied to industrial tasks such as the production line of a paint factory. The aim is to optimize the picking task with respect to manual operation in a paint factory. The use of an agile autonomous robot co-worker reduces the time in the picking process of materials, and the reduction of t...

Human collaboration is more likely to lead to cognitive growth when all group-members are actively involved in the collaborative process. However, there are cases that intra- group relationships need support. In this paper, we present an autonomous robotic system designed to interact with a pair of children in a problem-solving setting, aiming to u...

The paper presents a presents a framework for fire extinguishing in an urban scenario by a team of aerial and ground robots. The system was developed for the Challenge 3 of the 2020 Mohamed Bin Zayed International Robotics Challenge (MBZIRC). The challenge required to autonomously detect, locate and extinguish fires in different floors of a buildin...

This paper presents DLL, a direct map-based localization technique using 3D LIDAR for its application to aerial robots. DLL implements a point cloud to map registration based on non-linear optimization of the distance of the points and the map, thus not requiring features, neither point correspondences. Given an initial pose, the method is able to...

We present a novel implementation of a Rock-Paper-Scissors (RPS) game interaction with a social robot. The framework is tailored to be computationally lightweight, as well as entertaining and visually appealing through collaboration with designers and animators. The fundamental gesture recognition pipeline employs a Leap motion device and two separ...

This paper presents a non-linear optimization method for trajectory planning in a marsupial robot configuration. Particularly, the paper addresses the planning problem of an unmanned aerial vehicle (UAV) linked to an unmanned ground vehicle (UGV) by means of a tether. The result is a collision-free trajectory for UAV and tether, assuming the UGV po...

In this paper, we present a method of communicating affects from a remote user through the telepresence robot Haru. In this preliminary work, we transform the traditional mode of communicating text messages with emojis in the smartphone domain to the robot domain through robomojis- a hardware rendition of emojis. First we analyze human affects and...

This paper presents an unprecedented set of data in a challenging underground environment: the visitable sewers of Barcelona. To the best of our knowledge, this is the first dataset involving ground and aerial robots in such scenario: the SIAR ground robot and the ARSI aerial platform. These platforms captured data from a great variety of sensors,...

Exploration of spatial processes, such as radioactivity or temperature is a fundamental task in many robotic applications. In the literature, robotic exploration is mainly carried out for applications where the environment is a priori known. However, for most real life applications this assumption often does not hold, specifically for disaster scen...

This article presents a detailed report of the systems developed by the Skyeye team for the 2020 Mohamed Bin Zayed International Robotics Challenge (MBZIRC). This biennial competition aims to encourage researchers to develop fully autonomous systems for state-of-the-art applications in the area of field robotics. In this work, the main hardware and...

The emergence and development of cognitive strategies for the transition from exploratory actions towards intentional problem-solving in children is a key question for the understanding of the development of human cognition. Researchers in developmental psychology have studied cognitive strategies and have highlighted the catalytic role of the soci...

Information gathering (IG) algorithms aim to intelligently select the mobile robotic sensor actions required to efficiently obtain an accurate reconstruction of a physical process, such as an occupancy map, a wind field, or a magnetic field. Recently, multiple IG algorithms that benefit from multi-robot cooperation have been proposed in the literat...

This paper addresses an Active Cooperative Perception problem for Networked Robots Systems. Given a team of networked robots, the goal is finding a target using their inherent uncertain sensor data. The paper proposes a particle filter to model the probability distribution of the position of the target, which is updated using detection measurements...

The SIAR platform is a six-wheeled ground robot with differential kinematic configuration and automatic width adjustment developed for the ECHORD++ Challenge on Urban Robotics: “Robots For The Inspection And The Clearance Of The Sewer Network In Cities”. This challenge proposes the development of a wireless robotic platform for long range inspectio...

In this paper we summarize the automatic defect inspection onboard the sewer inspection ground platform SIAR. We include a general overview of the software and hardware characteristics of our platform, making a special emphasis on the sensing devices and software systems that are used for defect inspection. The main detection algorithm makes use of...

Sewers represent a very important infrastructure of cities whose state should be monitored periodically. However, the length of such infrastructure prevents sensor networks from being applicable. In this paper, we present a mobile platform (SIAR) designed to inspect the sewer network. It is capable of sensing gas concentrations and detecting failur...

Information gathering (IG) algorithms aim to intelligently select a mobile sensor actions required to efficiently obtain an accurate reconstruction of a physical process, such as an occupancy map, or a magnetic field. Many recent works have proposed algorithms for IG that employ Gaussian processes (GPs) as underlying model of the process. However,...

In this work we present a bioinspired visual system sensor to estimate angular rates in unmanned aerial vehicles (UAV) using Neural Networks. We have conceived a hardware setup to emulate Drosophila's ocellar system, three simple eyes related to stabilization. This device is composed of three low resolution cameras with a similar spatial configurat...

During the last years, UAVs have proved to be an essential tool in the mapping industry. Furthermore, the next generation of UAVs is envisioned to work cooperatively, following the swarming/teaming concept. This article presents MGRAPH, a novel approach to generate an incremental mosaic in real time from an UAV swarm. The algorithm is based on the...

Many recent works have proposed algorithms for information gathering that benefit from multi-robot cooperation. However, most algorithms either employ discretization of the state and action spaces, which makes them computationally intractable for robotic systems with complex dynamics; or cannot deal with inter-robot restrictions like e.g. communica...

This work presents an approach for learning navigation behaviors for robots using Optimal Rapidly-exploring Random Trees (RRT\(^{*}\)) as the main planner. A new learning algorithm combining both Inverse Reinforcement Learning and RRT\(^{*}\) is developed to learn the RRT\(^{*}\)’s cost function from demonstrations. A comparison with other state-of...

This work presents an approach to learn path planning for robot social navigation by demonstration. We make use of Fully Convolutional Neural Networks (FCNs) to learn from expert's path demonstrations a map that marks a feasible path to the goal as a classification problem. The use of FCNs allows us to overcome the problem of manually designing/ide...

This work presents an approach to learn path planning for robot social navigation by demonstration. We make use of Fully Convolutional Neural Networks (FCNs) to learn from expert's path demonstrations a map that marks a feasible path to the goal as a classification problem. The use of FCNs allows us to overcome the problem of manually designing/ide...

Information gathering algorithms aim to intelligently select the robot actions required to efficiently obtain an accurate reconstruction of a physical process, such as an occupancy map, or a magnetic field. Many recent works have proposed algorithms for information gathering. However, these algorithms employ discretization of the state space, which...

This paper presents a methodology for mapping and localization of Unmanned Aerial Vehicles (UAVs) based on the integration of sensors from different modalities. Particularly, we integrate distance estimations to Ultra-Wideband (UWB) sensors and 3D point-clouds from RGB-D sensors. First, a novel approach for environment mapping is introduced, exploi...

Multi-robot teams can play a crucial role in many applications such as exploration, or search and rescue operations. One of the most important problems within the multi-robot context is path planning. This has been shown to be particularly challenging, as the team of robots must deal with additional constraints, e.g. inter-robot collision avoidance...

This paper presents an approach for learning robot navigation behaviors from demonstration using Optimal Rapidly-exploring Random Trees (RRT\(^{*}\)) as main planner. A new learning algorithm combining both Inverse Reinforcement Learning (IRL) and RRT\(^{*}\) is developed in order to learn the RRT\(^{*}\)’s cost function from demonstrations. This c...

This paper addresses the problem of teaching a robot interaction behaviors using the imitation learning paradigm. Particularly, the approach makes use of Gaussian Mixture Models (GMMs) to model the physical interaction of the robot and the person when the robot is teleoperated or guided by an expert. The learned models are integrated into a sample-...

Robot navigation in human environments is an active research area that poses serious challenges in both robot perception and actuation. Among them, social navigation and human-awareness have gained lot of attention in the last years due to its important role in human safety and robot acceptance. Several approaches have been proposed; learning by de...

Robots navigating in a social way should reason about people intentions when acting. For instance, in applications like robot guidance or meeting with a person, the robot has to consider the goals of the people. Intentions are inherently non-observable, and thus we propose Partially Observable Markov Decision Processes (POMDPs) as a decision-making...

Rapidly exploring random trees (RRTs) have been proven to be efficient for planning in environments populated with obstacles. These methods perform a uniform sampling of the state space, which is needed to guarantee the algorithm's completeness but does not necessarily lead to the most efficient solution. In previous works it has been shown that th...

Social and Affective Robotics is a growing multidisciplinary field encompassing computer science, engineering, psychology, education, and many other disciplines. It explores how social and affective factors influence interactions between humans and robots, and how affect and social signals can be sensed and integrated into the design, implementatio...

Most localization approaches do not take into account the possibility of controlling the robot to improve the perception, instead, the robot is just commanded with a predefined path. Active sensing strategies may lead to more efficient exploration and mapping approaches. The robot can adapt its trajectory, avoiding for instance non-observable motio...

Exploration is a crucial problem in safety of life applications, such as search and rescue missions. Gaussian processes constitute an interesting underlying data model that leverages the spatial correlations of the process to be explored to reduce the required sampling of data. Furthermore, multi-agent approaches offer well known advantages for exp...

This chapter discusses architectures for multi-UAV cooperation. It focuses mainly on systems in which the UAVs have a certain degree of autonomy and that present intentional cooperation (thus swarming techniques and low-level formations are not covered). The chapter discusses the main elements of such architectures, with special emphasis on coopera...

The presence of children in a social assistive robotics context is particularly challenging for perception, mainly, in the task of locating them using inherently uncertain sensor data. This paper proposes a method for active perception with the goal of finding one target, e.g., a child wearing a RFID tag. This method is based on a particle-filter m...

Surveillance is an interesting application for Unmanned Aerial Vehicles (UAVs). If a team of UAVs is considered, the objective is usually to act cooperatively to gather as much information as possible from a set of moving targets in the surveillance area. This is a decision-making problem with severe uncertainties involved: relying on imperfect sen...

Target tracking with bearing-only sensors is a challenging problem when the target moves dynamically in complex scenarios. Besides the partial observability of such sensors, they have limited field of views, occlusions can occur, etc. In those cases, cooperative approaches with multiple tracking robots are interesting, but the different sources of...

TERESA is a socially intelligent semi-autonomous telepresence system that is currently being developed as part of an FP7-STREP project funded by the European Union. The ultimate goal of the project is to deploy this system in an elderly day centre to allow elderly people to participate in social events even when they are unable to travel to the cen...

This paper proposes extending Monte Carlo Localization methods with visual place recognition information in order to build a robust robot localization system. This system is aimed to work in crowded and non-planar scenarios, where 2D laser rangefinders may not always be enough to match the robot position within the map. Thus, visual place recogniti...

This chapter deals with the application of cooperative unmanned aerial systems to forest fires. Fire detection and fire monitoring and measurement to assist in fire extinguishing are discussed. The chapter presents a decision and control architecture for multi-UAS teams in forest firefighting. Then, the applications to fire detection and fire monit...

Planning under uncertainty faces a scalability problem when considering multi-robot teams, as the information space scales exponentially with the number of robots. To address this issue, this paper proposes to decentralize multi-robot Partially Observable Markov Decision Processes (POMDPs) while maintaining cooperation between robots by using POMDP...

This paper proposes extending Monte Carlo Localization methods with visual information in order to build a long term robot localization system. This system is aimed to work in crowded and non-planar scenarios, where 2D laser rangefinders may not always be enough to match the robot position with the map. Thus, visual place recognition will be used i...

An interesting applications of Unmanned Aerial Systems is surveillance. Surveillance typically involves the tracking of one or several targets in an area. A key issue for this application is the autonomous decision-making to allocate UAS to targets and determine the actions to be performed by the UAS of the fleet. Since this optimal decision-making...

This video details the development of an intelligent outdoor Guide robot. The main objective is to deploy an innovative robotic guide which is not only able to show information, but to react to the affective states of the users, and to offer location-based services using augmented reality. The scientific challenges concern autonomous outdoor naviga...

This video details the development of an intelligent outdoor guide robot. The main objective is to deploy an innovative robotic guide which is not only able to show information, but to react to the affective states of the users, and to offer location-based services using augmented reality. The scientific challenges concern autonomous outdoor naviga...

The paper considers a guiding task in which a robot has to guide a person towards a destination. A robust operation requires to consider uncertain models on the person motion and intentions, as well as noise and occlusions in the sensors employed for the task. Partially Observable Markov Decision Processes (POMDPs) are used to model the task. The p...

Robot navigation in human environments is an active research area that poses serious challenges. Among them, human-awareness has gain lot of attention in the last years due to its important role in human safety and robot acceptance. The proposed robot navigation system extends state of the navigation schemes with some social skills in order to natu...

In this paper, we introduce an approach called FSBS (Forward Search in Belief Space) for online planning in POMDPs. The approach is based on the RTBSS (Real-Time Belief Space Search) algorithm of [1]. The main departure from the algorithm is the introduction of similarity measures in the belief space. By considering statistical divergence measures,...

There is a clear trend in the use of robots to accomplish services that can help humans. In this paper, robots acting in urban environments are considered for the task of person guiding. Nowadays, it is common to have ubiquitous sensors integrated within the buildings, such as camera networks, and wireless communications like 3G or WiFi. Such infra...