Carlos Balaguer’s research while affiliated with University Carlos III de Madrid and other places

Omnidirectional mobile robots are essential in modern industry because of their ability to move in any direction, facilitating operations in confined spaces with high precision. This paper presents the design and development of an omnidirectional moving platform designed to transport loads on flat industrial surfaces, with three degrees of freedom that optimize its mobility. The mechatronic design combines kinematic analysis, motion control development, and a robust structure to improve the robot’s performance. A speed reduction system is incorporated to increase wheel torque, enabling efficient load handling. In addition, a control algorithm and a command device ensure precise and versatile operation. A prototype is developed and validated through practical tests that confirm its effectiveness in industrial environments. Areas of improvement are also identified for future optimizations, aligned with the growing demands of automation. This article highlights the importance of omnidirectional robots as a key solution for applications requiring precise mobility, efficiency and adaptability.

Large Language Models (LLMs) have substantially improved the conversational capabilities of social robots. Nevertheless, for an intuitive and fluent human-robot interaction, robots should be able to ground the conversation by relating ambiguous or underspecified spoken utterances to the current physical situation and to the intents expressed non verbally by the user, for example by using referential gaze. Here we propose a representation integrating speech and gaze to enable LLMs to obtain higher situated awareness and correctly resolve ambiguous requests. Our approach relies on a text-based semantic translation of the scanpath produced by the user along with the verbal requests and demonstrates LLM's capabilities to reason about gaze behavior, robustly ignoring spurious glances or irrelevant objects. We validate the system across multiple tasks and two scenarios, showing its generality and accuracy, and demonstrate its implementation on a robotic platform, closing the loop from request interpretation to execution.

Robotic manipulators are highly valuable tools that have become widespread in the industry, as they can achieve great precision and velocity in pick and place as well as processing tasks. However, to unlock their complete potential, some problems such as inverse kinematics (IK) need to be solved: given a Cartesian target, a method is needed to find the right configuration for the robot to reach that point. Another issue that needs to be addressed when dealing with robotic manipulators is the obstacle avoidance problem. Workspaces are usually cluttered and the manipulator should be able to avoid colliding with objects that could damage it, as well as with itself. Two alternatives exist to do this: a controller can be designed that computes the best action for each moment given the manipulator’s state, or a sequence of movements can be planned to be executed by the robot. Classical approaches to all these problems, such as numeric or analytical methods, can produce precise results but take a high computation time and do not always converge. Learning-based methods have gained considerable attention in tackling the IK problem, as well as motion planning and control. These methods can reduce the computational cost and provide results for every situation avoiding singularities. This article presents a literature review of the advances made in the past five years in the use of Deep Neural Networks (DNN) for IK with regard to control and planning with and without obstacles for rigid robotic manipulators. The literature has been organized in several categories depending on the type of DNN used to solve the problem. The main contributions of each reference are reviewed and the best results are presented in summary tables.

This paper explores the application of Inverse Reinforcement Learning (IRL) in robotics, focusing on inferring reward functions from expert demonstrations of robot arm manipulation tasks. By leveraging IRL, we aim to develop efficient and adaptable techniques for learning robust solutions to complex tasks in continuous state spaces. Our approach combines Apprenticeship Learning via IRL with Proximal Policy Optimization (PPO), expert-trajectory-based features, and the application of a reverse discount. The feature space is constructed by sampling expert trajectories to capture essential task characteristics, enhancing learning efficiency and generalizability by concentrating on critical states. To prevent the vanishing of feature expectations in goal states, we introduce a reverse discounting application to prioritize feature expectations in final states. We validate our methodology through experiments in a simple GridWorld environment, demonstrating that reverse discounting enhances the alignment of the agent’s features with those of the expert. Additionally, we explore how the parameters of the proposed feature definition influence performance. Further experiments on robotic manipulation tasks using the TIAGo robot compare our approach with state-of-the-art methods, confirming its effectiveness and adaptability in complex continuous state spaces across diverse manipulation tasks.

Today, it is of vital importance to research actuators and its control to harness their advantages, such as the force-to-weight ratio of the actuator or its energy consumption, in a more efficient way. For this reason, this work describes the characterization and control of actuators based on Shape Memory Alloy wires with different diameters ranging from 0.5 to 2 mm. These Nickel Titanium wires contract in length when heated, which makes them a distinguishing property compared to other alloys. This contraction is used to apply force so that it can be used in applications such as moving objects, controlling valves, or designing exoskeletons, among others. In this context, the objective of this work is to characterize Shape Memory Alloy wires of varying diameters, ranging from 0.5 mm to 2 mm, which are capable of lifting loads exceeding 100 kg with a single wire, while maintaining controlled positioning. Among the parameters to be highlighted are cooling and heating times, speed, energy consumption, breaking limits, and duty cycles. Additionally, various heating methods for Shape Memory Alloys wires are analyzed: direct heating by Joule effect and external heating. The results obtained with the proposed control strategy for steady-state position control (when the output remains within ±5% of the final value) were as follows: RMSE of 0.0401% for the one-way 0.51 mm wire, 0.0483% for the two-way 0.51 mm wire, 0.289% for the 1 mm wire, and 0.0857% for the 2 mm wire.

Robotic-assisted upper limb rehabilitation has gained significant attention in recent years due to its potential to enhance the recovery process for individuals with motor impairments resulting from neurological conditions and injuries. The main rehabilitation treatments rely on the repetitive execution of a movement of the upper-limb, guided by a therapist to prevent incorrect movements and to provide the necessary support. Many of the exercises performed by therapists can be modeled as a movement in SE(3) space (position and orientation). This movement itself is one-dimensional, as it can be modeled by a one-dimensional curve. To solve a similar problem, some approaches have been proposed in human-robot interaction (HRI) following virtual guides, but are either limited to specific types of curves (e.g. without orientation) or rely on linear control methods with non-intuitive parameters. To address these limitations and enable the use of these methods in physical rehabilitation, this paper extends Cartesian impedance control to splines, which we term path impedance control. It capitalizes on the intrinsic path geometry of end-effector robotic rehabilitation systems. The primary objective of this control algorithm is to emulate the sensation of maneuvering a physical object along a wire, akin to conventional exercise machines; and, in conjunction, provide an intuitive parametrization of rehabilitation exercises. We build on existing virtual guide control strategies using non-linear control and Lie Groups to generalize the control law to any one-parameter SE(3) curve.

This paper introduces a novel approach to robotic assistance in bottle opening using the dual-arm robot TIAGo++. The solution enhances accessibility by addressing the needs of individuals with injuries or disabilities who may require help with common manipulation tasks. The aim of this paper is to propose a method involving vision, manipulation, and learning techniques to effectively address the task of bottle opening. The process begins with the acquisition of bottle and cap positions using an RGB-D camera and computer vision. Subsequently, the robot picks the bottle with one gripper and grips the cap with the other, each by planning safe trajectories. Then, the opening procedure is executed via a position and force control scheme that ensures both grippers follow the unscrewing path defined by the cap thread. Within the control loop, force sensor information is employed to control the vertical axis movements, while gripper rotation control is achieved through a Deep Reinforcement Learning (DRL) algorithm trained to determine the optimal angle increments for rotation. The results demonstrate the successful training of the learning agent. The experiments confirm the effectiveness of the proposed method in bottle opening with the TIAGo++ robot, showcasing the practical viability of the approach.

To cite this article: Mas, J., Victores, J. G., 2024. Reinforcement Learning with the humanoid TEO. Jornadas de Automática, 45. https://doi.org/ Resumen En losúltimos años, el aprendizaje por refuerzo en entornos de simulación robótica ha emergido como una herramienta valiosa para entrenar plataformas robóticas en la ejecución de tareas complejas, como la marcha. El aprendizaje por refuerzo permite al robot descubrir un camino viable para realizar una tarea previamente definida, eliminando la necesidad de una pro-gramación exhaustiva y un control detallado de los movimientos. El propósito de este trabajo es mostrar la implementación de algoritmos de aprendizaje por refuerzo con el objetivo de conseguir que nuestro modelo del robot humanoide TEO aprenda a caminar sin necesidad de programar un controlador de manera explicita. Este artículo incluye como se ha desarrollado el modelo del humanoide, que medida de aprendizaje se ha desarrollado y que algoritmos se han implementado durante el entrenamiento, así como los resultados que se han obtenido de este entrenamiento. Palabras clave: Aprendizaje por refuerzo y aprendizaje profundo en control, Aprendizaje automático en modelado, predicción, control y automatización, Guía, navegación y control, Arquitectura del software de control, Sistemas robóticos autónomos Optimization of Walking with Reinforcement Learning in Humanoid TEO Abstract In recent years, reinforcement learning in robotic simulation environments has emerged as a valuable tool for training robotic platforms to perform complex tasks, such as walking. Reinforcement learning allows the robot to discover a viable path to perform a predefined task, eliminating the need for exhaustive programming and detailed control of movements. The purpose of this work is to demonstrate the implementation of reinforcement learning algorithms with the aim of enabling our humanoid robot model TEO to learn to walk without the need for explicit controller programming. This paper includes the development of the humanoid model, the learning metric developed, and the algorithms implemented during training, as well as the results obtained from this training. Keywords: Reinforcement learning and deep learning in control, Machine learning in modelling, prediction, control and automation, Guidance navigation and control, Control software architecture, Autonomous robotic systems 1. Introducción La coordinación y eficiencia en la ejecución de tareas por parte de robots continúan siendo desafíos complejos en la robótica. Los robots humanoides, en particular, enfrentan ma-yores dificultades debido a su estructura intrincada y la nece-sidad de imitar movimientos humanos. El robot humanoide TEO, con una altura de 1,65 metros y 28 grados de libertad, ilustra estos desafíos. La programa-ción y el control de los movimientos necesarios para realizar cualquier tarea con este robot son extremadamente complejos. Esta complejidad se debe, en parte, al alto número de grados de libertad y a la dificultad inherente en la coordinación y es-tabilidad de todos los movimientos del humanoide. En el artículo de (Radosavovic et al., 2024) se presenta un enfoque completamente basado en el aprendizaje para la lo