No file available
To read the file of this research,
you can request a copy directly from the authors.
Real-time Dexterous Telemanipulation with an End-Effect-Oriented Learning-based Approach
Preprints and early-stage research may not have been peer reviewed yet.
Abstract
Dexterous telemanipulation is crucial in advancing human-robot systems, especially in tasks requiring precise and safe manipulation. However, it faces significant challenges due to the physical differences between human and robotic hands, the dynamic interaction with objects, and the indirect control and perception of the remote environment. Current approaches predominantly focus on mapping the human hand onto robotic counterparts to replicate motions, which exhibits a critical oversight: it often neglects the physical interaction with objects and relegates the interaction burden to the human to adapt and make laborious adjustments in response to the indirect and counter-intuitive observation of the remote environment. This work develops an End-Effects-Oriented Learning-based Dexterous Telemanipulation (EFOLD) framework to address telemanipulation tasks. EFOLD models telemanipulation as a Markov Game, introducing multiple end-effect features to interpret the human operator's commands during interaction with objects. These features are used by a Deep Reinforcement Learning policy to control the robot and reproduce such end effects. EFOLD was evaluated with real human subjects and two end-effect extraction methods for controlling a virtual Shadow Robot Hand in telemanipulation tasks. EFOLD achieved real-time control capability with low command following latency (delay<0.11s) and highly accurate tracking (MSE<0.084 rad).
ResearchGate has not been able to resolve any citations for this publication.
In real world applications, robotic solutions remain impractical due to the challenges that arise in unknown and unstructured environments. To perform complex manipulation tasks in complex and cluttered situations, robots need to be able to identify the interaction possibilities with the scene, i. e. the affordances of the objects encountered. In unstructured environments with noisy perception, insufficient scene understanding and limited prior knowledge, this is a challenging task. In this work, we present an approach for grasping unknown objects in cluttered scenes with a humanoid robot in the context of a nuclear decommissioning task. Our approach combines the convenience and reliability of autonomous robot control with the precision and adaptability of teleoperation in a semi-autonomous selection of grasp affordances. Additionally,this allows exploiting the expert knowledge of an experienced human worker. To evaluate our approach, we conducted 75 real world experiments with more than 660 grasp executions on the humanoid robot ARMAR-6. The results demonstrate that high-level decisions made by the human operator, supported by autonomous robot control, contribute significantly to successful task execution.
We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies that can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed in a simulated environment in which we randomize many of the physical properties of the system such as friction coefficients and an object’s appearance. Our policies transfer to the physical robot despite being trained entirely in simulation. Our method does not rely on any human demonstrations, but many behaviors found in human manipulation emerge naturally, including finger gaiting, multi-finger coordination, and the controlled use of gravity. Our results were obtained using the same distributed RL system that was used to train OpenAI Five. We also include a video of our results: https://youtu.be/jwSbzNHGflM .
The correct grasp of objects is a key aspect for the right fulfillment of a given task. Obtaining a good grasp requires algorithms to automatically determine proper contact points on the object as well as proper hand configurations, especially when dexterous manipulation is desired, and the quantification of a good grasp requires the definition of suitable grasp quality measures. This article reviews the quality measures proposed in the literature to evaluate grasp quality. The quality measures are classified into two groups according to the main aspect they evaluate: location of contact points on the object and hand configuration. The approaches that combine different measures from the two previous groups to obtain a global quality measure are also reviewed, as well as some measures related to human hand studies and grasp performance. Several examples are presented to illustrate and compare the performance of the reviewed measures.
This paper presents a fiducial marker system specially appropriated for camera pose estimation in applications such as augmented reality and robot localization. Three main contributions are presented. First, we propose an algorithm for generating configurable marker dictionaries (in size and number of bits) following a criterion to maximize the inter-marker distance and the number of bit transitions. In the process, we derive the maximum theoretical inter-marker distance that dictionaries of square binary markers can have. Second, a method for automatically detecting the markers and correcting possible errors is proposed. Third, a solution to the occlusion problem in augmented reality applications is shown. To that aim, multiple markers are combined with an occlusion mask calculated by color segmentation. The experiments conducted show that our proposal obtains dictionaries with higher inter-marker distances and lower false negative rates than state-of-the-art systems, and provides an effective solution to the occlusion problem.
The purpose of this technical report is two-fold. First of all, it introduces a suite of challenging continuous control tasks (integrated with OpenAI Gym) based on currently existing robotics hardware. The tasks include pushing, sliding and pick & place with a Fetch robotic arm as well as in-hand object manipulation with a Shadow Dexterous Hand. All tasks have sparse binary rewards and follow a Multi-Goal Reinforcement Learning (RL) framework in which an agent is told what to do using an additional input. The second part of the paper presents a set of concrete research ideas for improving RL algorithms, most of which are related to Multi-Goal RL and Hindsight Experience Replay.
In shared autonomy, user input is combined with semi-autonomous control to achieve a common goal. The goal is often unknown ex-ante, so prior work enables agents to infer the goal from user input and assist with the task. Such methods tend to assume some combination of knowledge of the dynamics of the environment, the user's policy given their goal, and the set of possible goals the user might target, which limits their application to real-world scenarios. We propose a deep reinforcement learning framework for model-free shared autonomy that lifts these assumptions. We use human-in-the-loop reinforcement learning with neural network function approximation to learn an end-to-end mapping from environmental observation and user input to agent action, with task reward as the only form of supervision. Controlled studies with users (n = 16) and synthetic pilots playing a video game and flying a real quadrotor demonstrate the ability of our algorithm to assist users with real-time control tasks in which the agent cannot directly access the user's private information through observations, but receives a reward signal and user input that both depend on the user's intent. The agent learns to assist the user without access to this private information, implicitly inferring it from the user's input. This allows the assisted user to complete the task more effectively than the user or an autonomous agent could on their own. This paper is a proof of concept that illustrates the potential for deep reinforcement learning to enable flexible and practical assistive systems.
Deep reinforcement learning is poised to revolutionise the field of AI and represents a step towards building autonomous systems with a higher level understanding of the visual world. Currently, deep learning is enabling reinforcement learning to scale to problems that were previously intractable, such as learning to play video games directly from pixels. Deep reinforcement learning algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of reinforcement learning, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep reinforcement learning, including the deep Q-network, trust region policy optimisation, and asynchronous advantage actor-critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via reinforcement learning. To conclude, we describe several current areas of research within the field.
Shared autonomy integrates user input with robot autonomy in order to control a robot and help the user to complete a task. Our work aims to improve the performance of such a human-robot team: the robot tries to guide the human towards an effective strategy, sometimes against the human's own preference, while still retaining his trust. We achieve this through a principled human-robot mutual adaptation formalism. We integrate a bounded-memory adaptation model of the human into a partially observable stochastic decision model, which enables the robot to adapt to an adaptable human. When the human is adaptable, the robot guides the human towards a good strategy, maybe unknown to the human in advance. When the human is stubborn and not adaptable, the robot complies with the human's preference in order to retain their trust. In the shared autonomy setting, unlike many other common human-robot collaboration settings, only the robot actions can change the physical state of the world, and the human and robot goals are not fully observable. We address these challenges and show in a human subject experiment that the proposed mutual adaptation formalism improves human-robot team performance, while retaining a high level of user trust in the robot, compared to the common approach of having the robot strictly following participants' preference.
In this work a novel synergy-based bilateral tele-manipulation strategy is introduced. The proposed algorithm has been primarily developed to remotely control the Pisa/IIT SoftHand (SH) using a 3-finger hand exoskeleton as master device. With a single actuator and a sensory system limited to a position encoder and a current sensor, the SH minimalist design promotes robustness but challenges traditional teleoperation strategies. To tackle this challenge, the concept of Cartesian-based hand synergies is introduced as a projection tool which maps the fingertip Cartesian space to the directions oriented along the grasp principal components. The unconstrained motion of the operator's hand is projected on this space to extract the SH's motor position reference. Conversely, the interaction force estimated at the robotic hand as a 1-dimensional force along the first synergy is projected to the 9D fingertip Cartesian space through an inverse projection. The resultant finger-individualized forces form a synergy based weighted representation of the grasping effort applied by the SH and are displayed to the operators fingertips using the force feedback hand exoskeleton. The system's ability to reflect the environment's impedance has been experimentally validated during a ball squeezing experiment. To assess the overall effectiveness of the proposed system as a manipulation interface, the SoftHand was mounted on the humanoid robot COMAN and the setup was subsequently enriched with a vision-based tracking system monitoring the operators wrist trajectory. Experimental results indicate that the proposed body-machine bilateral interface allows for the intuitive performance of stable grasps and transport of a large range of diversely shaped objects.
Teleoperation systems have been among the first applications of the robotic technology, mainly because of the interest for the nuclear research back in the 50's. Since then, dramatic developments have been achieved, both from the technological point of view (materials, computers, possibility of building miniaturized devices, ...) and from the theoretical one. In this paper, attention is given in particular to the control aspects, and an overview on the main schemes that have been presented in the literature is presented, suggesting some criteria for their analysis and comparison.
Solving rubik's cube with a robot hand
- I Openai
- M Akkaya
- Andrychowicz
OpenAI, I. Akkaya, M. Andrychowicz, et al., "Solving rubik's cube
with a robot hand," ArXiv, vol. abs/1910.07113, 2019.
Markov games as a framework for multi-agent reinforcement learning
- M L Littman
M. L. Littman, "Markov games as a framework for multi-agent
reinforcement learning," in Machine learning proceedings 1994,
Elsevier, 1994, pp. 157-163.
Mapping human to robot motion with functional anthropomorphism for teleoperation and telemanipulation with robot arm hand systems
- M Liarokapis
- P K Artemiadis
- K J Kyriakopoulos
M. V Liarokapis, P. K. Artemiadis, and K. J. Kyriakopoulos, "Mapping
human to robot motion with functional anthropomorphism for
teleoperation and telemanipulation with robot arm hand systems," in
2013 IEEE/RSJ International Conference on Intelligent Robots and
Systems, IEEE, 2013, p. 2075.
Markov decision processes
- F Garcia
- E Rachelson
F. Garcia and E. Rachelson, "Markov decision processes," Markov
Decis. Process. Artif. Intell., pp. 1-38, 2013.
Gen6D: Generalizable model-free 6-DoF object pose estimation from RGB images
- Y Liu
Y. Liu et al., "Gen6D: Generalizable model-free 6-DoF object pose
estimation from RGB images," in European Conference on Computer
Vision, Springer, 2022, pp. 298-315.
Tactile sensing and deep reinforcement learning for in-hand manipulation tasks
- A Melnik
- L Lach
- M Plappert
- T Korthals
- R Haschke
- H Ritter
A. Melnik, L. Lach, M. Plappert, T. Korthals, R. Haschke, and H. Ritter,
"Tactile sensing and deep reinforcement learning for in-hand
manipulation tasks," in IROS Workshop on Autonomous Object
Manipulation, 2019.
An objectbased approach to map human hand synergies onto robotic hands with dissimilar kinematics
- G Gioioso
- G Salvietti
- M Malvezzi
- D Prattichizzo
G. Gioioso, G. Salvietti, M. Malvezzi, and D. Prattichizzo, "An objectbased approach to map human hand synergies onto robotic hands with
dissimilar kinematics," Robot. Sci. Syst. VIII, pp. 97-104, 2012.