Fuqiang Zhou’s research while affiliated with Beihang University and other places

Inner walls detection of precision pipelines with large length-to-diameter ratios is an important task in industrial production, including the measurement of geometric parameters. However, current vision measurement methods have limitations in such confined spaces, suffering from large volume, high cost, low precision and efficiency, leading to difficulties in meeting requirements for measuring straight and long pipelines. To address these issues, we propose a stereo vision sensor based on catadioptric units with planar mirrors in obtaining characteristics of modularity and scalability, which can achieve multi-directional detection in confined spaces such as pipelines. The sensor is composed of multiple catadioptric units arranged symmetrically around the central axis, forming multi-directional virtual binocular imaging. The geometric model of the catadioptric unit is established for quantitative analysis of performance indicators, including measurement precision, OPD, DOF and FOV. The optimized parameters are selected to construct a prototype sensor, which is used for experimental verification of pipeline inner diameter measurement. The results show that the sensor can achieve high-precision and stable measurement. Through expanding and stacking multiple proposed sensors with modularity, the field-of-view will be increased to improve detection efficiency. The combination with pipeline robots will provide new solutions for the measurement and detection of the inner walls of precision pipelines with large length-to-diameter ratios.

Environmental perception, which is crucial for robotic decision-making and execution control, presents significant challenges in developing lightweight compact, low-cost, real-time and flexible three-dimensional vision sensor with Moderate measurement accuracy. Compared with stereo-vision sensor based-on multi-camera, single-camera mirrored binocular sensor provides more compact structure and lower cost. Furthermore, the single camera mirrored binocular has the advantage of real-time measurement compared with the actual binocular realized by moving camera. This paper develops a mirrored binocular vision sensor with single-plane mirror and only one camera. A single-plane mirror is used to form a virtual binocular, and the structural parameters of the sensor are optimized from the aspects of spatial arrangement, field-of-view (FOV), measurement accuracy and depth-of-field (DOF). Based on a planar target calibration approach, the camera calibration accuracy is 0.064 pixels, and the average calibration error of the sensor is 0.022 mm. Utilizing deep learning algorithm, the operational performance in both structured and unstructured environments is validated through experiments, and the results demonstrate that the accuracy of 3-dimensional (3D) reconstruction is greater than 0.1 mm.

In 3D reconstruction based on multi-line structured light stereo vision sensors, identifying the laser stripes and light planes from binocular images is a fundamental yet challenging problem. However, the existing methods are ineffective for images exhibiting large-area deformation, breakage, and disappearance of the laser stripes, which will result in the failure or inefficiency of 3D reconstruction. This paper solves this problem by identifying the laser stripes via ray-model. We first calibrate the ray emitted by the laser based on ray-model and build a pixel-ray lookup table. In the identification process, the lookup table and epipolar line are employed to identify the potential matching points. Subsequently, the laser stripes and light planes are identified through the distance constraint between the 3D points and the light planes. The efficacy of the proposed method is evaluated through a comparative experiment with existing methods, and the average recognition rate reaches 99.5% through multiple experiments.

Defect detection of inner surface of precision pipes is a crucial aspect of ensuring production safety. Currently, pipeline defect detection primarily relies on recording video for manual recognition, with urgent need to improve automation, quantification and accuracy. This paper presents a hexapod in-pipe robot with carrying capacity designed to transport the omnidirectional vision sensor to specified location within unreachable pipelines. The feasibility of the robot’s mechanical design and sensor load-carrying module is analyzed using theory calculations, motion simulations and finite element method. To address the challenges of small pixel ratio and weak background changes in panoramic images, a tiny defect segmentor based on ResNet is proposed for detecting tiny defects on the inner surface of pipelines. The hardware and software systems are implemented, and the motion performance of the pipeline robot is validated through experiments. The results demonstrate that the robot achieves stable movement at a speed of over 0.1 m s⁻¹ and can adapt to pipe diameter ranging from of 110 to 130 mm. The novelty of the robot lies in providing stable control of the loaded vision sensor, with control precision of the rotation angle and the displacement recorded at 1.84% and 0.87%, respectively. Furthermore, the proposed method achieves a detection accuracy of 95.67% for tiny defects with a diameter less than 3 mm and provides defect location information. This pipeline robot serves as an essential reference for development of in-pipe 3D vision inspection system.