No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Design and Control of Continuous Gait for Humanoid Robots: Jumping, Walking, and Running Based on Reinforcement Learning and Adaptive Motion Functions
ResearchGate has not been able to resolve any citations for this publication.
Bipedal gait control has always been a very challenging issue due to the multi‐joint and non‐linear structure of humanoid robots and frequent robot–environment interactions. To realise stable and robust bipedal walking, many aspects including robot modelling, gait stability and environmental adaptivity should be considered to design the gait control method. In this paper, a general description of bipedal gait and the corresponding evaluation indicators are introduced. Moreover, the existing bipedal gait control methods are classified into model‐based gait, stability criterion‐based gait and learning strategy‐based gait and a comprehensive review is conducted. Finally, the existing challenges and development trends of bipedal gait control are presented.
Maintaining stability and completing the required trajectory are the most important motion control issues for a six-kinematic-degrees-of-freedom legged robot. Zero-moment point (ZMP) approaches have evolved into an indispensable tool for obtaining dynamic equilibrium in humanoid or bipod robots for walking and handling. It's point where momentary, applied, and reactive forces cancel out. Despite ZMP advancements, legged robots continue to struggle with a number of essential skills exhibited by primates, such as like the Atlas robot, which can walk but not jump, run, or sprint. In complex environments, it may be incapable of interception. Hence, in this study, the effect of center of mass (CoM) trajectory on biped robot stability has been explored to understand the secure walking behavior. The research investigates the predictive control technique of adjusting CoM trajectory to generate stable walking patterns, which pioneered the concept of altering the CoM in advance during human walking and reacting to changing path restrictions. This section explains how to fix continuous variation's ZMP tracking problem. This allows monitoring the necessary ZMP, expanding gait generation systems and their analysis. Predictive control can produce secure walking models by manipulating the COM and CoG pathline in bipod robots.
The planning of whole-body motion and step time for bipedal locomotion is constructed as a model predictive control (MPC) problem, in which a sequence of optimization problems needs to be solved online. While directly solving these problems is extremely time-consuming, we propose a predictive gait synthesizer to offer immediate solutions. Based on the full-dimensional model, a library of gaits with different speeds and periods is first constructed offline. Then the proposed gait synthesizer generates real-time gaits at 1kHz by synthesizing the gait library based on the online prediction of centroidal dynamics. We prove that the constructed MPC problem can ensure the uniform ultimate boundedness (UUB) of the CoM states and show that our proposed gait synthesizer can provide feasible solutions to the MPC optimization problems. Simulation and experimental results on a bipedal robot with 8 degrees of freedom (DoF) are provided to show the performance and robustness of this approach.
Bipedal walking is one of the most difficult but exciting challenges in robotics. The difficulties arise from the complexity of high-dimensional dynamics, sensing and actuation limitations combined with real-time and computational constraints. Deep Reinforcement Learning (DRL) holds the promise to address these issues by fully exploiting the robot dynamics with minimal craftsmanship. In this paper, we propose a novel DRL approach that enables an agent to learn omnidirectional locomotion for humanoid (bipedal) robots. Notably, the loco-motion behaviors are accomplished by a single control policy (a single neural network). We achieve this by introducing a new curriculum learning method that gradually increases the task difficulty by scheduling target velocities. In addition, our method does not require reference motions which facilities its application to robots with different kinematics, and reduces the overall complexity. Finally, different strategies for sim-to-real transfer are presented which allow us to transfer the learned policy to a real humanoid robot.
Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms, however, have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This article addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multiagent deep RL (MADRL) is presented, including nonstationarity, partial observability, continuous state and action spaces, multiagent training schemes, and multiagent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to the future development of more robust and highly useful multiagent learning methods for solving real-world problems.
According to many, we are at the brink of the fourth industrial revolution. The theme of Industry 4.0 is "Smart Manufacturing for the Future". Now, some futurists even discuss what the fifth industrial revolution’s theme will be. There are a few visions for Industry 5.0. One emerging theme is human-robot co-working. In recent years, we have seen significant advancements in robotics and artificial intelligence (AI) research. Today, there are robots for various purposes at affordable prices in the market. It is not long before we closely interact with robots in our lives and workplaces. Testing autonomous cars in traffic is a promising example of this upcoming trend. There are companies having an employee record for robots or AI applications. While there are many studies on human-robot collaboration for low-level tasks with a focus on robot development, we lack studies focusing on organizational issues emerging from human-robot co-working. In this study, we discuss the possible issues related to human-robot co-working from the organizational and human employee’s perspective. We believe the issues identified in this study will be the focus of many upcoming organizational robotics research studies.
Recently, the method of choice to exploit robot dynamics for efficient walking is numerical optimization (NO). The main drawback in NO is the computational complexity, which strongly affects the time demand of the solution. Several strategies can be used to make the optimization more treatable and to efficiently describe the solution set. In this work, we present an algorithm to encode effective walking references, generated off-line via numerical optimization, extracting a limited number of principal components and using them as a basis of optimal motions. By combining these components, a good approximation of the optimal gaits can be generated at run time. The advantages of the presented approach are discussed, and an extensive experimental validation is carried out on a planar legged robot with elastic joints. The biped thus controlled is able to start and stop walking on a treadmill, and to control its speed dynamically as the treadmill speed changes.
This article introduces a method incorporating the gait and feedback controller design for a four-link biped robot. Inspired by the human gait and previous biped work, we propose a class of periodic gaits composed from the sum of sinusoidal harmonics up to the second order. This class of gaits, parameterized by design parameters, satisfies the impact model of the biped dynamics and enables us to design the step length and average walking velocity explicitly. The satisfaction of the impact model introduces constraints on design parameters, which together with the explicit parameterization significantly reduces the number of the design variables. Compared to the hybrid zero dynamics (HZD) method, the process of the gait design requires fewer parameters and shorter computation time, and thus, it can be conducted in real-time systems. The state-feedback controller is then proposed to ensure that the periodic gait is the limit cycle of the closed-loop system and its stability is theoretically guaranteed based on the Poincaré section method. Due to the explicit parameterization, the controller can be easily extended to achieve walking on different slopes with the same step length and average walking velocity. Finally, the theoretical result is verified by simulations for biped walking on different slopes with various gaits and controller parameters. The controller is found to be robust to external disturbance and model uncertainty.
Teleoperation of humanoid robots enables the integration of the cognitive skills and domain expertise of humans with the physical capabilities of humanoid robots. The operational versatility of humanoid robots makes them the ideal platform for a wide range of applications when teleoperating in a remote environment. However, the complexity of humanoid robots imposes challenges for teleoperation, particularly in unstructured dynamic environments with limited communication. Many advancements have been achieved in the last decades in this area, but a comprehensive overview is still missing. This survey article gives an extensive overview of humanoid robot teleoperation, presenting the general architecture of a teleoperation system and analyzing the different components. We also discuss different aspects of the topic, including technological and methodological advances, as well as potential applications.
This paper presents a trajectory generation algorithm with which a three-dimensional (3D) biped robot can perform aperiodic gaits by modifying only a small set of gait parameters. In addition to the double support phase (DSP), the gait can transit smoothly from one single support phase (SSP) to another. We decouple the sagittal and coronal dynamics, firstly. In the sagittal plane, the linear inverted pendulum model (LIPM) is used to generate the walking reference trajectory during the SSP. An extra template model, linear pendulum model (LPM), is added to describe the motion in the DSP. For the coronal plane, we only utilize the LPM for trajectory generation. Thanks to linearity properties, the proposed method can obtain the biped locomotion computationally fast without the need for numerical time-integration. Herein, we also introduce the trajectory generation algorithm for different aperiodic gaits including from standing to walking, stopping walking, as well as speed switch. A full-dynamics 3D humanoid robot is used for the tests of the developed reference trajectory through simulation. The results are promising for implementations.
In this article, we holistically present a hybrid-linear inverted pendulum (H-LIP) based approach for synthesizing and stabilizing 3-D foot-underactuated bipedal walking, with an emphasis on thorough hardware realization. The H-LIP is proposed to capture the essential components of the underactuated and actuated part of the robotic walking. The robot walking gait is then
directly
synthesized based on the H-LIP. We comprehensively characterize the periodic orbits of the H-LIP and provably derive the stepping stabilization via its step-to-step (S2S) dynamics, which is then utilized to approximate the S2S dynamics of the horizontal state of the center of mass of the robotic walking. The approximation facilities a H-LIP based stepping controller to provide desired step sizes to stabilize the robotic walking. By realizing the desired step sizes, the robot achieves dynamic and stable walking. The approach is fully evaluated in both simulation and experiment on the 3-D underactuated bipedal robot Cassie, which demonstrates dynamic walking behaviors with both high versatility and robustness.
Passive dynamic walking of bipeds exhibits a new concept of robot locomotion that is well- known for its success in human mimicry. Many efforts have been devoted in the literature to implement flexible and anthropomorphic passive bipeds. However, utilization of flexible characteristics of elastic structures has not been considered yet. This paper presents a modified simplest passive walking model with flexible legs which are a pair of elastic beams with small deflections instead of usual rigid links. The main contributions of this paper lie in modeling the impulsive hybrid dynamics of the flexible passive biped and applying a numerical technique based on finite difference formula to seek for appropriate initial conditions of walking gait cycles. Extended Hamilton's principle and assumed mode method along with Euler–Bernoulli's beam theory and updated transition rules at impact are used to derive the mathematical model of this biped. Numerical simulations reveal non-periodic patterns of motion for small slope angles due to vibrations of elastic legs. It is also shown that the period-one gait cycles with quicker steps can be generated on slower slopes for different values of elasticity and damping coefficients compared to the rigid model.
Deep reinforcement learning (RL) has become one of the most popular topics in artificial intelligence research. It has been widely used in various fields, such as end-to-end control, robotic control, recommendation systems, and natural language dialogue systems. In this survey, we systematically categorize the deep RL algorithms and applications, and provide a detailed review over existing deep RL algorithms by dividing them into modelbased methods, model-free methods, and advanced RL methods. We thoroughly analyze the advances including exploration, inverse RL, and transfer RL. Finally, we outline the current representative applications, and analyze four open problems for future research.
Humanoid robots generated by inspiring by human appearances and abilities have became essential in human society to improve the quality of their life. All over the world, there have been many researchers who have focused on humanoid robots to develop the capabilities of humanoid robots. Generally, humanoid robot systems include mechanisms of decision making and information processing. Because of the uncertainty behind decision making and information processes, fuzzy sets are used most commonly. This study investigates a comprehensive literature review about humanoid robots that presents the recent technological developments and the theories associated with fuzzy set models. The basic principles and concepts of fuzzy sets for humanoid robots are presented.
The development of a versatile, fully-capable humanoid robot as envisioned in science fiction books is one of the most challenging but interesting issues in the robotic field. Currently, existing humanoid robots are designed with different purposes and applications in mind. In humanoid robot development process, each robot is designed with various characteristics, abilities, and equipment, which influence the general structure, cost, and difficulty of development. Even though humanoid robot development is very popular, a few review papers are focusing on the design and development process of humanoid robots. Motivated by this, we present this review paper to show variations in the requirements, design, and development process and also propose a taxonomy of existing humanoid robots. It aims at demonstrating a general perspective of existing humanoid robots’ characteristics and applications. This paper includes state-of-the-art and successfully reported existing humanoid robot designs along with different robots used in various robot competitions.
Gait generation plays a decisive role to the performance of biped robot walking. Under mathematical viewpoint, the task of gait generation design is investigated as an optimization problem with respect to various trade-off constraints, hence it prefers to evolutionary computation techniques. This paper introduces a novel approach for the biped robot gait generation which aims to enable humanoid robot to walk more naturally and stably on flat platform. The proposed modified differential evolution (MDE) optimisation algorithm is initiatively applied to optimally identify the novel adaptive evolutionary neural model (AENM) for a dynamic biped gait generator. The performance of proposed MDE method is demonstrated in comparison with other genetic algorithm (GA) and particle swarm optimisation (PSO) approaches. The proposed method is implemented and tested on a prototype small-sized humanoid robot. The identification result demonstrates that the new proposed neural AENM model proves an effective approach for a robust and precise biped gait generation.
With the development of Industry 4.0, robots tend to be intelligent and collaborative. For one, robots can interact naturally with humans. For another, robots can work collaboratively with humans in a common area. The traditional teaching method is no longer suitable for the production mode with human-robot collaboration. Since the traditional teaching processes are complicated, they need highly skilled staffs. The paper focuses on the natural way of online teaching, which can be applied to the tasks such as welding, painting and stamping. This paper presents an online teaching method with the fusion of speech and gesture. A depth camera (Kinect) and an inertial measurement unit (IMU) are used to capture the speech and gesture of the human. Interval Kalman filter and improved particle filter are employed to estimate the gesture. To integrate speech and gesture information more deeply, a novel method of audio-visual fusion based on text (AVFT) is proposed, which can extract the most useful information from the speech and gestures by transforming them into text. IEEE
From knowing to doing: learning diverse motor skills through instruction learning
- L Ye