Maodong Ren’s research while affiliated with Shenzhen China Star Optoelectronics Technology Co., Ltd and other places

In single camera stereo digital image correlation systems, the existing calibration methods have problems with poor accuracy and violation of theoretical constraints, so we have proposed a hybrid calibration method. In this study, we first explain the principles of the existing single camera stereo measurement calibration methods. Then, on the basis of the classical camera calibration method combined with the characteristics of planar mirror reflection imaging, a hybrid calibration method is proposed that not only satisfies the constraints of the same intrinsic parameters but also has higher accuracy. Next, we complete the mathematical model of the hybrid calibration method and explain the specific calibration process. Finally, the experimental results of point distance reconstruction and static displacement measurement show that, compared with the existing calibration methods, the hybrid calibration method has lower error and higher reconstruction accuracy.

Industrial measurements increasingly employ a redundant robotic system with an external turntable due to its efficiency, flexibility, and automation advantages. However, the actual viewpoint pose reached by the robotic system deviates from its simulated design pose due to orientation errors, resulting in poor measurement quality of complex surfaces. Geometry calibration is necessary for reducing robot orientation errors. However, most calibration methods ignore the factors of multiple error coupling and robotic absolute positioning error propagation, ultimately affecting calibration accuracy. In order to resolve this problem, this paper proposes a robust self-calibration method for the simultaneous identification of geometric pose parameters. First, considering the overall synchronized control of orientation errors, a comprehensive error transfer model is derived as the theoretical basis. Then, the error model receives the system orientation errors as input, which are detected from a multi-view visual measurement process on a calibration panel. Finally, the error model is solved using a self-adaptive Levenberg-Marquardt (SALM) algorithm, and the identified error vectors are used for online compensation of the kinematic model, improving the orientation accuracy of the robotic system. Simulations and experiments verify the accuracy and robustness of the proposed method. The experiment results show that the RMSE detection deviation of the measuring point cloud compensated by self-calibration is reduced by 96.9% from 2.215 to 0.068 mm compared with before calibration. This self-calibration method is simple, economical, and general and can be further extended to other multi-axis automation systems.

To achieve high-precision and high-efficiency 3D scanning of industrial parts, this research proposes an automated 3D scanning system based on path planning algorithms, in which the 3D scanning sensor is realized based on the principle of raster projection. The proposed automated scanning system is composed of a robotic arm, a 3D scanner, and a turntable to achieve a complete 3D scanning of the object to be measured. More precisely, this paper proposes a virtual 3D scanning principle, which can accurately determine whether a certain point on the CAD model can be scanned by a viewpoint. Further, this paper proposes a path planning algorithm based on the virtual 3D scanning principle, which can calculate the optimal scanning path required to scan the object according to the CAD model of the scanned object. Furthermore, to ensure the accuracy of the system, a high-precision system calibration method is proposed to establish the accurate positional relationship between various parts of the system. Firstly, this research proposes a camera calibration algorithm based on close-range photogrammetry, which can calculate the high-precision interior and exterior orientation parameters of the camera, and apply the method of calculating camera external parameters to the hand-eye calibration, making the accuracy of hand-eye calibration is also improved accordingly. In addition, this research proposes a photogrammetry-based projector calibration algorithm to determine the positional relationship between the projector and the camera in the 3D scanner, and a turntable calibration method to determine the positional relationship between the rotation center of the turntable and the camera. Eventually, according to different usage requirements, two 3D scanning data splicing methods are proposed. One is a high-efficient automatic splicing method based on the results of system calibration without pasting mark points, and the other is a high-precision data splicing method based on photogrammetry. Experimental results demonstrate that this system has higher accuracy and better performance in system calibration, scanning path calculation, and multi-view scanning data splicing.

The digital manufacturing deviation inspection can be used to assess the dimension, form, and position errors, which is more efficient, more consistent, and more robust than manual inspection. CAD models are used as the nominal data and compared with the scanned data of parts to obtain manufacturing errors. CAD model tessellation is a pivotal step in deviation inspection, which directly determines the reliability of analytical results. It requires that the generated triangle mesh has dense vertices, high approximation accuracy, and consistent orientation. A single surface patch tessellation method is proposed, and the adaptive quadtree is first constructed to divide the domain into small spatial grids in the parameter space of the patch; then, a clip-based method is presented to triangulate boundary grids, and inner grids could be triangulated by the constrained Delaunay triangulation method. Furthermore, the mesh patches should be spliced and reoriented consistently, which is very important for deviation analysis. To solve this problem, a robust gap healing method based on merging boundary points of patches is developed, so that all boundary vertices of adjacent patches could completely coincide along the common edges, and all mesh patches would be stitched into a complete mesh without gaps. Finally, the face normal of mesh is adjusted to obtain a consistent orientation. Experiments on industrial CAD models demonstrate that the proposed method could realize faster and more robust mesh generation than the state-of-the-art approaches, including three commercial and two open-source software.

The full-field and non-contact measurement of three-dimensional motion and deformation of high-speed rotating machinery is a very challenging task in the engineering field. Two issues, the cost of high-speed camera and the decorrelation between rotated images, are hard to avoid for investigators. In this paper, a systematic solution is presented, which combines high-speed rotation image acquisition using low-speed camera with 3D digital image correlation, both low-cost, flexible and accurate. Firstly, the frequency and initial phase of high-speed rotation are calculated by FFT, and temporal phase-locked signal is generated to trigger the dual-camera asynchronously to collect the circumferential images with equal phase interval. Then a layer-by-layer matching method is proposed to search uncoded targets between two rotated images, while a binocular rotation correlating strategy to correlate the speckle subsets. Finally, the corresponded uncoded targets and speckle subsets are reconstructed in three dimension via the camera calibration results, furtherly the 3D motion and deformation fields can be obtained. Five experiments, including two measurement repeatability tests, an accuracy test, a rigid motion measurement of rotating table and a dynamic deformation measurement of ventilation fan, prove the accuracy and effectiveness of the proposed method.

In the triangular meshes obtained in additive repair, it is a challenge to find one single hole-filling method to close all holes and make the filling patches assort with surrounding meshes well with low time complexity, which is mainly caused by the shape complexity and size difference of the various holes, especially in the fields of intelligent manufacturing, 3D measurement, and reverse engineering. Therefore, it is reasonable to adopt different algorithms to fill different types of holes. In this research, a fast hole-filling method for triangular mesh is proposed based on the hole size. First, a group of basic concepts is defined to make them uniform throughout the whole text, followed by the descriptions of hole detection and boundary cleaning. Second, three different algorithms are developed to fill the small-sized, middle-sized, and large-sized holes classified by hole size respectively, which can fill all the detected holes in a fast and proper manner. Finally, two experiments are carried out to verify the efficiency, robustness, and ability to recover the shape of our method. Compared to two state-of-the-art hole-filling methods in the first experiment, the quantitative evaluation results demonstrate that our proposed method is much faster than them with the ability to guarantee the regularity of most filling triangles. The second experiment proves that our method can produce satisfactory filling results by making the filling patches be compatible with surrounding meshes well.

Due to the huge number of points on three-dimensional point clouds captured by optical scanning devices, point-based simplification is a crucial step in model reconstruction. However, the loss of edge features of industrial parts after such simplification reduces reconstruction accuracy. This paper presents an edge-sensitive, point-based simplification method to eliminate redundant points and preserve more edge feature details. Firstly, a new geometrical descriptor is created for each point to generate a geometrical domain. A clustering scheme is then designed by applying two different clustering algorithms, to split the point cloud in the geometrical domain and the spatial domain respectively. The proposed method is capable of preserving edge features well, while reducing the original number of points to 10% or even 5%. The proposed method is compared with other simplification methods and the experimental results indicate that it performs better in simplifying industrial parts.

Reconstructing 3D human body models for real people benefits a variety of applications. However, it is challenging to reconstruct high-fidelity human body models with high-accuracy. This study presents an accurate and efficient 3D human body reconstruction system based on digital image correlation (DIC), which can simultaneously capture high-quality 3D human body model and high-fidelity 2D color texture. The 3D reconstruction system is set up with twenty-seven cameras and nine speckle-pattern projectors, which is able to acquire 3D shape data and 2D color data. Specifically, the nine binocular stereo vision systems are adopted to reconstruct 3D shape, where the 3D reconstruction is carried out based on stereo-DIC with a speckle-pattern projector. The other nine color cameras are utilized to capture 2D color texture. It only needs a single-shot to record the images required for 3D reconstruction, which takes about 1.25 ms to ensure the high-accuracy and high-efficiency of the proposed 3D reconstruction system. A high-precision multi-camera calibration method is employed to calibrate the proposed multi-camera system. The mapping relationship between the reconstructed 3D object point and the 2D color image is determined by the calibrated result, which makes the color texture acquisition precisely and effectively. The proposed system can achieve accurate and high-quality 3D reconstruction of the whole human body surface with high-fidelity color texture. Experimental results demonstrate the high-performance of the proposed 3D reconstruction system.

In this paper, a videogrammetric system, based on three-dimensional digital image processing and industrial close range photogrammetry technology, is proposed to solve the three-dimensional motion and deformation parameters of a rotating blade by identifying a speckle pattern. For the high aspect ratio and flexible rotor blade design often used in heavy-duty helicopters, there are extremely complex aerodynamic, dynamic, and coupling problems that require solutions to measure the full-field deformation of the rotating blade in the test condition of a rotor test tower. The key issues involved include a two-step calibration method using separate internal and external parameters for a large field of view, and a reliable estimation of the seed point method for a large deformation and weak correlation speckle images match. The system was developed and a simulation test was conducted on a rotor test tower. The results show that the proposed system is practicable to measure the surface displacement and motion parameters of a high-speed rotating rotor blade.

An accurate and efficient 3-Dinterpolation scheme, based on sampling theorem and Fourier transform technique, is proposed to reduce the sub-voxel matching error caused by intensity interpolation bias in digital volume correlation. First, the influence factors of the interpolation bias are investigated theoretically using the transfer function of an interpolation filter (henceforth filter) in the Fourier domain. A law that the positional error of a filter can be expressed as a function of fractional position and wave number is found. Then, considering the above factors, an optimized B-spline-based recursive filter, combining B-spline transforms and least squares optimization method, is designed to virtually eliminate the interpolation bias in the process of sub-voxel matching. Besides, given each volumetric image containing different wave number ranges, a Gaussian weighting function is constructed to emphasize or suppress certain of wave number ranges based on the Fourier spectrum analysis. Finally, a novel software is developed and series of validation experiments were carried out to verify the proposed scheme. Experimental results show that the proposed scheme can reduce the interpolation bias to an acceptable level.