Figure 10 - available via license: CC BY
Content may be subject to copyright.
Source publication
Mobile manipulation has a broad range of applications in robotics. However, it is usually more challenging than fixed-base manipulation due to the complex coordination of a mobile base and a manipulator. Although recent works have demonstrated that deep reinforcement learning is a powerful technique for fixed-base manipulation tasks, most of them a...
Context in source publication
Similar publications
Autonomous and Robotics Systems (ARSs) are widespread, complex, and increasingly coming into contact with the public. Many of these systems are safety-critical, and it is vital to detect software errors to protect against harm. We propose a family of novel techniques to detect unusual program executions and incorrect program behavior. We model exec...
This paper presents a team of multiple Unmanned Aerial Vehicles (UAVs) to perform cooperative missions for autonomous construction. In particular, the UAVs have to build a wall made of bricks that need to be picked and transported from different locations. First, we propose a novel architecture for multi-robot systems operating in outdoor and unstr...
Exploration is a fundamental problem in robot autonomy. A major imitation, however, is that during exploration robots oftentimes have to rely on on-board systems alone for state estimation, accumulating significant drift over time in large environments. Drift can be detrimental to robot safety and exploration performance. In this work, a submap-bas...
Citations
... The time efficiency of MM can be improved by jointly solving navigation and manipulation tasks. For example, by using whole-body (base & manipulator) motion control [3], [4], [5], [6], [7], determining optimal base pose for grasping [8], [9], [10], or by jointly planning base pose and pre-grasp manipulator configuration [11], [12]. The whole-body motion control methods attempt to directly reach the desired End-Effector (EE) pose by combining base & manipulator motion. ...
In Mobile Manipulation (MM), navigation and manipulation are generally solved as subsequent disjoint tasks. Combined optimization of navigation and manipulation costs can improve the time efficiency of MM. However, this is challenging as precise object pose estimates, which are necessary for such combined optimization, are often not available until the later stages of MM. Moreover, optimizing navigation and manipulation costs with conventional planning methods using uncertain object pose estimates can lead to failures and hence requires replanning. Hence, in the presence of object pose uncertainty, preactive approaches are preferred. We propose such a pre-active approach for determining the base pose and pre-grasp manipulator configuration to improve the time efficiency of MM. We devise a Reinforcement Learning (RL) based solution that learns suitable base poses for grasping and pre-grasp manipulator configurations using layered learning that guides exploration and enables sample-efficient learning. Further, we accelerate learning of pre-grasp manipulator configurations by providing dense rewards using a predictor network trained on previously learned base poses for grasping. Our experiments validate that in the presence of uncertain object pose estimates, the proposed approach results in reduced execution time. Finally, we show that our policy learned in simulation can be easily transferred to a real robot. The code repository and the supplementary video can be found on the project webpage*.
... Learning [100] Mobile manipulation is difficult because of the complex coordination of the mobile base and manipulator. ...
This article presents a literature review of the past five years of studies using Deep Reinforcement Learning (DRL) and Inverse Reinforcement Learning (IRL) in robotic manipulation tasks. The reviewed articles are examined in various categories, including DRL and IRL for perception, assembly, manipulation with uncertain rewards, multitasking, transfer learning, multimodal, and Human-Robot Interaction (HRI). The articles are summarized in terms of the main contributions, methods, challenges, and highlights of the latest and relevant studies using DRL and IRL for robotic manipulation. Additionally, summary tables regarding the problem and solution are presented. The literature review then focuses on the concepts of trustworthy AI, interpretable AI, and explainable AI (XAI) in the context of robotic manipulation. Moreover, this review provides a resource for future research on DRL/IRL in trustworthy robotic manipulation.
... However, this method has a limitation in fully utilizing a mobile manipulator. The second approach considers a mobile manipulator as a single high-degree-of-freedom system [9][10]. However, the high computational cost of this method makes it challenging to perform other tasks such as online obstacle avoidance. ...
... They used the RRT-GoalBias algorithm and postprocessed the initial path to smoothen it. In [10], motion planning was performed using a reinforcement learning approach based on the proximal policy optimization (PPO) algorithm for grasping objects. However, the high computational cost of this method makes it difficult to perform other tasks such as online obstacle avoidance. ...
... As a result, the work is expressed by (9), representing the value at position 2 in the VIEF. Because is formulated in a quadratic form for position 2 , the minimum value of at 2, can be computed by differentiating with respect to 2 as shown in (10). In (11) confirms that when there are no external forces and dampers in the system, 2, becomes , which resembles a potential field. ...
Motion planning for mobile manipulators is challenging because of their high degrees of freedom. The most effective approach for the motion planning of a mobile manipulator is to consider the different characteristics of a mobile robot and a manipulator while planning and controlling each system separately. In a previous study, different characteristics were considered using virtual impedance. This method involves forming a virtual impedance relationship between the two subsystems, enabling the mobile robot to track the movement of the manipulator. However, this study had certain limitations. Firstly, this method is not applicable to non-holonomic mobile robots. Secondly, obstacle avoidance methods for mobile robots are not considered. To address these limitations, we propose a novel concept for our motion planner that is called the virtual impedance energy field (VIEF), which refers to the work and change in total energy. We solved the first limitation of the previous study by transforming the virtual impedance force into the VIEF. Moreover, by integrating an obstacle field with the VIEF and using it as a local costmap, the second limitation was addressed. To validate the performance of our motion planner, we conducted simulations in environments comprising obstacles, with straight and curved trajectories of the end-effector. We then analyzed the pose change graph of the mobile robot and end-effector, pose error of the end-effector, and velocity of the mobile robot. We therefore confirmed that the manipulator successfully followed the desired trajectory, while the mobile robot maintained a distance from the end-effector and avoided obstacles.
... In recent years, deep learning has been introduced into path planning problems, and good results have been obtained. [4] Compared to deep reinforcement learning, Value Iteration Networks provide better performance on never-before-seen maps and enhanced generalization capabilities. [5] VINet embedded a "planning program" in the deep reinforcement neural network, so that the neural network can learn not only from the state to the decision of direct mapping, but also can learn how to make long-term planning, assisting the neural network and long-term planning is used to make better decisions. ...
Motion Planning is a key technology for mobile robots, which decomposes a Motion task that cannot be completed by a single action into multiple discrete actions that can be performed. This paper aims to design a robot motion planning algorithm based on reinforcement learning and make a robot carry out continuous multi-objective point motion planning. Motion planning network is a planning algorithm based on a neural network, and DQN is a classical algorithm in the field of reinforcement learning. Based on the two kinds of algorithm for motion planning, the Deep Q - learning algorithm chooses the robot’s next target, and then through the motion planning of the network between the current coordinates to the next target path planning. This paper analyzes the performance of the multi-point motion planning algorithm, and the results show that the algorithm is able to a higher success rate of successful completion of the task planning, but the reward strategy derived from the experiment still has the possibility of optimization.
... Other works focus on tabletop rearrangement [28,21,12] while our work focuses on longer horizon mobile manipulation. Although mobile manipulation has witnessed significant achievements in robotics learning [29,43,26,30,19], due to the vastly increased complexity, in rearrangement tasks, a majority of previous works [6,40] explicitly separate the base and arm movements to simplify the task. A few works [16,47,45] have shown the superiority of mobile manipulation over static manipulation. ...
We present Skill Transformer, an approach for solving long-horizon robotic tasks by combining conditional sequence modeling and skill modularity. Conditioned on egocentric and proprioceptive observations of a robot, Skill Transformer is trained end-to-end to predict both a high-level skill (e.g., navigation, picking, placing), and a whole-body low-level action (e.g., base and arm motion), using a transformer architecture and demonstration trajectories that solve the full task. It retains the composability and modularity of the overall task through a skill predictor module while reasoning about low-level actions and avoiding hand-off errors, common in modular approaches. We test Skill Transformer on an embodied rearrangement benchmark and find it performs robust task planning and low-level control in new scenarios, achieving a 2.5x higher success rate than baselines in hard rearrangement problems.
... However, designing planning and control systems for highly complex systems such as mobile manipulators is a challenge for conventional methods, as they are sensitive to parameters and require significant engineering effort [11]. To address this issue, reinforcement learning (RL) has emerged as a promising approach for complex robot control [12]- [14] including mobile manipulators [15]- [18]. As it can be less sensitive to hardware calibration [19], generate robot actions directly from low-level sensor observations, and learn the policy by itself without much prior knowledge and fine-tuning [20], it is suitable for tasks such as door opening. ...
The ability of robots to navigate through doors is crucial for their effective operation in indoor environments. Consequently, extensive research has been conducted to develop robots capable of opening specific doors. However, the diverse combinations of door handles and opening directions necessitate a more versatile door opening system for robots to successfully operate in real-world environments. In this paper, we propose a mobile manipulator system that can autonomously open various doors without prior knowledge. By using convolutional neural networks, point cloud extraction techniques, and external force measurements during exploratory motion, we obtained information regarding handle types, poses, and door characteristics. Through two different approaches, adaptive position-force control and deep reinforcement learning, we successfully opened doors without precise trajectory or excessive external force. The adaptive position-force control method involves moving the end-effector in the direction of the door opening while responding compliantly to external forces, ensuring safety and manipulator workspace. Meanwhile, the deep reinforcement learning policy minimizes applied forces and eliminates unnecessary movements, enabling stable operation across doors with different poses and widths. The RL-based approach outperforms the adaptive position-force control method in terms of compensating for external forces, ensuring smooth motion, and achieving efficient speed. It reduces the maximum force required by 3.27 times and improves motion smoothness by 1.82 times. However, the non-learning-based adaptive position-force control method demonstrates more versatility in opening a wider range of doors, encompassing revolute doors with four distinct opening directions and varying widths.
... A second group of methods directly uses reinforcement learning over the entire action space to learn a policy. This enables the robot to utilize simultaneously all of its actuation capabilities but, when applied directly, is limited in the complexity of tasks and environments it can handle [52], with performance quickly deteriorating as tasks get more complex [17]. More complex MoMa tasks can be tackled by Honerkamp et al. [12], who learn base trajectories to adapt to given MoMa trajectories. ...
... Multi-task robotic learning has also been approached from the perspective of learning to reach goals [5,49,27,19], as well as learning policies that can perform tasks in a discrete set or some other parameterized form [7,8,13,29]. Mobile manipulation policy learning too has been studied, primarilary via reinforcement-learning [32,26,67,37,65,20,18,11,69], and recently through foundation models [3,38,66]. Herein, we add to this line of work, where we provide evidence that it is possible to learn simple mobile manipulation tasks in an end-to-end fashion with large-scale Transformer policies. ...
... Prior robot learning methods have achieved success in mobile manipulation in simulation [47][48][49] or the real world [50][51][52][53][54][55][56][57][58][59]. Some of these methods collect data in a continuous, online manner [50,52,53], while others break data collection into primitives [54,55] to learn to perform long-horizon mobile manipulation. ...
In this paper, we propose a method to create visuomotor mobile manipulation solutions for long-horizon activities. We propose to leverage the recent advances in simulation to train visual solutions for mobile manipulation. While previous works have shown success applying this procedure to autonomous visual navigation and stationary manipulation, applying it to long-horizon visuomotor mobile manipulation is still an open challenge that demands both perceptual and compositional generalization of multiple skills. In this work, we develop Mobile-EMBER, or M-EMBER, a factorized method that decomposes a long-horizon mobile manipulation activity into a repertoire of primitive visual skills, reinforcement-learns each skill, and composes these skills to a long-horizon mobile manipulation activity. On a mobile manipulation robot, we find that M-EMBER completes a long-horizon mobile manipulation activity, cleaning_kitchen, achieving a 53% success rate. This requires successfully planning and executing five factorized, learned visual skills.
... A second group of methods directly uses reinforcement learning over the entire action space to learn a policy. This enables the robot to utilize simultaneously all of its actuation capabilities but, when applied directly, is limited in the complexity of tasks and environments it can handle [52], with performance quickly deteriorating as tasks get more complex [17]. More complex MoMa tasks can be tackled by Honerkamp et al. [12], who learn base trajectories to adapt to given MoMa trajectories. ...
Developing the next generation of household robot helpers requires combining locomotion and interaction capabilities, which is generally referred to as mobile manipulation (MoMa). MoMa tasks are difficult due to the large action space of the robot and the common multi-objective nature of the task, e.g., efficiently reaching a goal while avoiding obstacles. Current approaches often segregate tasks into navigation without manipulation and stationary manipulation without locomotion by manually matching parts of the action space to MoMa sub-objectives (e.g. base actions for locomotion objectives and arm actions for manipulation). This solution prevents simultaneous combinations of locomotion and interaction degrees of freedom and requires human domain knowledge for both partitioning the action space and matching the action parts to the sub-objectives. In this paper, we introduce Causal MoMa, a new framework to train policies for typical MoMa tasks that makes use of the most favorable subspace of the robot's action space to address each sub-objective. Causal MoMa automatically discovers the causal dependencies between actions and terms of the reward function and exploits these dependencies in a causal policy learning procedure that reduces gradient variance compared to previous state-of-the-art policy gradient algorithms, improving convergence and results. We evaluate the performance of Causal MoMa on three types of simulated robots across different MoMa tasks and demonstrate success in transferring the policies trained in simulation directly to a real robot, where our agent is able to follow moving goals and react to dynamic obstacles while simultaneously and synergistically controlling the whole-body: base, arm, and head. More information at https://sites.google.com/view/causal-moma.
