Xuebo Zhang

Nankai University | NKU · Institute of Robotics and Automatic Information System

Illumination change, image blur, and fast motion dramatically decrease the performance of visual-inertial navigation systems (VINS). This article presents a new illumination-robust visual-inertial odometry (IR-VIO) based on adaptive weighting algorithm with two-layer confidence maximization. First, to prevent the VIO performance degradation caused...

This paper proposes a lightweight and efficient Neighborhood Encoding-based Global Localization (NEGL) approach for unmanned ground vehicles (UGVs). To realize the reliable feature description and overcome the restriction of limited Field of View (FOV), neighborhood encoding (NE) scheme is firstly proposed to describe the feature via constructing t...

In this article, we propose a safe visibility-guided perception-aware trajectory optimization method for aerial tracking, which can handle occlusion and collision in complex environments simultaneously. Different from the existing methods, the key idea of the proposed method is to actively adjust the relative distance and angle between the quadroto...

The basic prerequisite of visual servoing control for a quadrotor unmanned aerial vehicle (UAV) is to ensure that the visual targets are always within the field of view (FOV) of the camera. In this paper, a new geometric control approach is proposed under FOV constraint for the quadrotor, in which the control barrier function (CBF) is used to const...

In this paper, a fisheye-based visual servo regulation method is proposed for wheeled mobile robots, which utilizes motion planning algorithms to provide trajectories for driving the robot to desired poses in an aligned pattern. To satisfy the wide field-of-view (FOV) requirement in robotic systems, the image projection model of fisheye is analyzed...

Autonomous agile flight control has been a challenging problem due to complex highly nonlinear dynamics, and generating feasible basic flight maneuvers off-line for subsequent online motion planning has become a solution. In this paper, we present a novel reinforcement learning-based basic flight maneuvers generation method for a 6 degrees of freed...

Robotic grasping aims to detect graspable points and their corresponding gripper configurations in a particular scene, and is fundamental for robot manipulation. Existing research works have demonstrated the potential of using a transformer model for robotic grasping, which can efficiently learn both global and local features. However, such methods...

In this paper, an enhanced visual servoing method is designed for a quadrotor unmanned aerial vehicle (UAV) based on virtual plane image moments, under underactuation and tight coupling constraints of UAV kinematics. Moreover, in order to make the UAV search visual targets autonomously in target vicinity during flight, a flexible flight system is d...

Motion planning is fundamental for the autonomous navigation of quadrotors in unknown environments, which often consists of path searching and trajectory generation. However, existing path searching approaches overlook the influence of the tortuosity of the path, and trajectory generation methods usually ignore optimizing the time allocation. They...

In this paper, a novel multi-robot autonomous exploration approach CURE is proposed based on dynamic Voronoi diagrams and centroids of unknown connected regions. Compared with existing approaches, the novelty of this work is twofold: 1) Dynamic Voronoi diagram is used for partition of the space being explored to improve the efficiency of multi-robo...

The paper proposes a novel visual servoing trajectory tracking controller satisfying velocity saturation constraints for mobile robots, which can simultaneously realize the unknown image depth identification. Compared with existing saturation controllers, the boundness of velocity commands can be explicitly determined though the control law is coup...

Monocular visual odometry (VO) is a fundamental technique for unmanned ground vehicle (UGV) navigation. However, traditional monocular VO methods always suffer from sparse environment maps which cannot be directly used for navigation because of the lack of structural information. In this article, we propose a navigation-oriented LiDAR-guided monocu...

Place recognition is an important problem in environment perception, which can help simultaneous localization and mapping (SLAM), as well as scene understanding of mobile robots. For place recognition, feature aggregation plays an important role in the representation of environments. In this paper, an adaptive soft-encoding (ASE) method is proposed...

This article proposes a compact vectorized representation approach to realize reliable obstacle detection for autonomous ground vehicles. Bridging the gap between the advantage of occupancy grid map in general obstacle detection and its shortcomings in memory-consuming and computational cost, a novel obstacle representation by multiple convex polyg...

The paper presents a novel method called D\(^2\)WA for local planning of mobile robots in indoor environments with moving obstacles. The first “dynamic” in the title has twofold meanings: (1) The proposed motion planning method is applied for dynamic environments, which is the key difference between D\(^2\)WA and the traditional dynamic window appr...

p>Illumination change, image blur, and fast motion dramatically decrease the performance of visual-inertial navigation systems (VINS). This paper presents a new illumination-robust visual-inertial odometry (IR-VIO) based on adaptive weighting algorithm with two-layer confidence maximization. First, to prevent the VIO performance degradation caused...

p>Illumination change, image blur, and fast motion dramatically decrease the performance of visual-inertial navigation systems (VINS). This paper presents a new illumination-robust visual-inertial odometry (IR-VIO) based on adaptive weighting algorithm with two-layer confidence maximization. First, to prevent the VIO performance degradation caused...

Guaranteeing the stability and fast transient performance of a direct-drive system simultaneously, which is practically subject to various modeling uncertainties, velocity, and input constraints, is both theoretically and technically challenging. To this end, an online time-optimal trajectory optimization-based saturated adaptive robust control alg...

To solve the autonomous navigation problem in complex environments, an efficient motion planning approach is newly presented in this paper. Considering the challenges from large-scale, partially unknown complex environments, a three-layer motion planning framework is elaborately designed, including global path planning, local path optimization, and...

This article proposes a novel asymptotic robust control approach for rotorcraft unmanned aerial vehicles (UAVs), which can effectively eliminate the impact of external disturbances and the model uncertainties. Different from existing works, the proposed method alleviates the assumption that disturbances should have no variations in the existing obs...

Dense depth estimation is significant in robotic systems, such as for mapping, localization, and object recognition. For multiple sensors, an active depth sensor can provide accurate but sparse measurements for environments, and a camera pair can provide dense but imprecise stereo reconstruction results. In this paper, a tightly coupled fusion meth...

The instantaneous large disturbances, which are typically characterized by large amplitudes and short durations, generally force the actual trajectory of a servo motor system to deviate from the desired trajectory dramatically. In view of this, a novel fast trajectory recovery scheme is proposed mainly from the perspective of trajectory planning. T...

According to the requirement of general static obstacle detection, this paper proposes a compact vectorization representation approach of local static environments for unmanned ground vehicles. At first, by fusing the data of LiDAR and IMU, high-frequency pose information is obtained. Then, through the two-dimensional (2D) obstacle points generatio...

With the rapid development of micro aerial vehicle (MAV) technology, autonomous coverage and exploration of interest environments with multi-rotor MAVs have become a research hotspot. Firstly, this article briefly introduces the research contents of coverage and exploration planning of interest environments by using robotics. Then, three aspects ar...

In this letter, an efficient motion planning approach with grid-based generalized Voronoi diagrams is newly proposed for mobile robots. Different from existing approaches, the novelty of this work is twofold: 1) a new state lattice-based path searching approach is proposed, in which the search space is reduced to a Voronoi corridor to further impro...

This article proposes an autonomous aerial exploration framework for traversable path finding of ground robots in unknown environments to achieve a fast response in search and rescue (SAR) missions. Different from existing works, our method provides a task-oriented active exploration strategy that makes the aerial robot fully autonomous during the...

Practical servo motor systems are generally subject to various modeling uncertainties and actuating constraints. It is challenging to guarantee a minimum-time transient response when modeling uncertainties and constraints present simultaneously. As such, a two-loop control algorithm, which integrates a smooth projection based saturated adaptive rob...

Multi-sensor fusion is a mainstream method for localization of unmanned systems. How to achieve 6-degrees of freedom (DOF) pose estimation of the system is challenging in GPS-denied environments. Although map-aided localization methods normally perform well on intelligent transportation systems, prior maps are unavailable in some GPS-denied scenes...

In this article, a novel informative autonomous exploration strategy with uniform sampling is proposed to efficiently reduce the unknown volume of 3D environments and provide accurate truncated signed distance function (TSDF) based reconstruction for quadrotors. Different from existing methods, the proposed method locally samples candidate viewpoin...

To solve the autonomous navigation problem in complex environments, an efficient motion planning approach is newly presented in this paper. Considering the challenges from large-scale, partially unknown complex environments, a three-layer motion planning framework is elaborately designed, including global path planning, local path optimization, and...

In this paper, a novel method for point cloud segmentation based on Euclidean clustering and multi-plane extraction is proposed. To cope with overhanging objects, such as tree branches, a hybrid elevation map assisted by Euclidean clustering is designed. By clustering the 3D point clouds falling into the grid cell, the obstacles above a free space...

For the visual odometry (VO) and simultaneous localization and mapping (SLAM) in indoor environments, high-level geometric features have attracted more and more attention in recent years. Unlike the generally used point features, the geometric features, such as planes and lines, encode more higher-level semantic information of the scene which is be...

To solve the autonomous navigation problem in complex environments, an efficient motion planning approach called EffMoP is presented in this paper. Considering the challenges from large-scale, partially unknown complex environments, a three-layer motion planning framework is elaborately designed, including global path planning, local path optimizat...

To solve the autonomous navigation problem in complex environments, an efficient motion planning approach is newly presented in this paper. Considering the challenges from large-scale, partially unknown complex environments, a three-layer motion planning framework is elaborately designed, including global path planning, local path optimization, and...

Local planning is one of the key technologies for mobile robots to achieve full autonomy and has been widely investigated. To evaluate mobile robot local planning approaches in a unified and comprehensive way, a mobile robot local planning benchmark called MRPB 1.0 is newly proposed in this paper. The benchmark facilitates both motion planning rese...

Owing to advantages of large workspace and flexible movement, wheeled mobile robots are widely applied in industry. With vision module for environment perception, visual servoing of wheeled mobile robots has been one of the hottest research topics in robotic fields. Due to different demands for various applications and specific properties of the sy...

For pose stabilization task of nonholonomic mobile robots, this article proposes a novel integrated interactive framework, bridging the gap between visual servoing and simultaneous localization and mapping (SLAM). The framework consists of two cooperative components, control module for servoing task and SLAM module for feedback signals estimation....

In this paper, an adaptive visual tracking controller is proposed for the nonholonomic mobile robot with an onboard monocular camera, wherein both the velocity and acceleration saturation constraints can be guaranteed by properly selecting the control parameters during the visual servoing process. Specifically, the virtual velocity-level controller...

In this paper, we propose a complete visual coverage trajectory planning framework for quadrotor Unmanned Aerial Vehicles (UAVs) in three-dimensional (3-D) terrain environment under photogrammetric constraints. To ensure the accuracy of the 3-D reconstruction, the air-to-ground projective geometric constraints are established. To optimize explorati...

In this article, a virtual-goal-guided rapidly exploring random tree (RRT)-based visual servoing approach is proposed for nonholonomic mobile robots to simultaneously satisfy the field-of-view (FOV) constraint and the velocity constraints during the motion toward the desired pose. The presented approach contains two parts: 1) trajectory planning in...

To solve the autonomous navigation problem in complex environments, an efficient motion planning approach called EffMoP is presented in this paper. Considering the challenges from large-scale, partially unknown complex environments, a three-layer motion planning framework is elaborately designed, including global path planning, local path optimizat...

In this paper, we propose a novel optimization-based tightly coupled Direct Visual-Inertial Odometry (DVIO), which fuses the visual and inertial measurements to provide real-time full state estimation. Different from existing frameworks, the key novelty of the proposed method is to integrate the data association, state estimation, and outlier detec...

Planes and edges are attractive features for simultaneous localization and mapping (SLAM) in indoor environments because they can be reliably extracted and are robust to illumination changes. However, it remains a challenging problem to seamlessly fuse two different kinds of features to avoid degeneracy and accurately estimate the camera motion. In...

Local planning is one of the key technologies for mobile robots to achieve full autonomy and has been widely investigated. To evaluate mobile robot local planning approaches in a unified and comprehensive way, a mobile robot local planning benchmark called MRPB 1.0 is newly proposed in this paper. The benchmark facilitates both motion planning rese...

Visual coverage is an important task for environment perception. In this paper, the coverage of a large-scale three-dimensional (3-D) scene represented by a polygon mesh model is considered, and a visual sensor network deployment algorithm is proposed through the combination of space partition, greedy and local search procedures. Comparing with exi...

A novel real-time time-optimal trajectory planning (TOTP) approach is proposed for rotorcrafts under velocity constraints and complex non-convex thrust, attitude input constraints. Different from existing works, the proposed method is the first real-time TOTP solution for rotorcrafts subject to velocity constraints and these non-convex input constr...

In this article, a novel real-time acceleration-continuous path-constrained trajectory planning algorithm is proposed with an appealing built-in tradeoff mechanism between the cruise motion and time-optimal motion. Different from existing approaches, the proposed approach smoothens time-optimal trajectories with bang-bang input structures to genera...

This paper proposes a novel adaptive image-based output feedback visual servoing approach to control a multirotor to the desired pose by using a minimum onboard sensor suite which consists of an inertial measurement unit and a monocular camera. Different from ‘perspective moment’ (PM), a new type of image feature is designed as ‘rotated perspective...

Visual place recognition is a challenging problem in simultaneous localization and mapping (SLAM) due to a large variability of the scene appearance. A place is usually described by a single-frame image in conventional place recognition algorithms. However, it is unlikely to completely describe the place appearance using a single frame image. Moreo...

Coverage is a basic and critical issue for design and deployment of visual sensor networks, however, the optimization problem is very challenging especially when considering coverage of three-dimensional (3-D) scenarios. This paper provides a brief survey of mainstream coverage optimization methods for visual sensor networks, including the greedy a...

This paper presents a feature classification method based on vision sensor in dynamic environment. Aiming at the detected targets, a double-projection error based on orb and surf is proposed, which combines texture constraints and region constraints to achieve accurate feature classification in four different environments. For dynamic targets with...

WiFi-based indoor localization has received extensive attentions from both academia and industry. However, the overhead of constructing and maintaining the WiFi fingerprint map remains a bottleneck for the wide-deployment of WiFi-based indoor localization systems. Recently, robots are adopted as the professional surveyor to fingerprint the environm...

Purpose This paper aims to automatically plan sequence for complex assembly products and improve assembly efficiency. Design/methodology/approach An assembly sequence planning system for workpieces (ASPW) based on deep reinforcement learning is proposed in this paper. However, there exist enormous challenges for using DRL to this problem due to th...

The paper presents derivation, control perfor mance analysis, and numerical validation of the Vector Field Orientation (VFO) control law devised for visual servoing of the input-constrained unicycle-like robot equipped with an on-board camera of an uncertain depth scale factor. It is shown that an inherent robustness of the proposed VFO controller...

Target recognition is a challenging task for human following of mobile service robots. In this paper, we combine the principal-component-analysis (PCA)-based face recognition with the tracking-learning-detection applied to the human face (Face-TLD) to obtain an improvement, named as IFace-TLD. The proposed IFace-TLD can significantly improve the di...