Project

Real-time 3D shape measurement

Goal: To realize real-time high-accuracy 3D shape measurement for intelligent manufacturing.

Updates
0 new
12
Recommendations
0 new
6
Followers
0 new
17
Reads
3 new
252

Project log

Jiaming Qian
added an update
The constructed tri-color SIM system. For more information, follow our new project: Structured illumination microscopy (https://www.researchgate.net/project/Structured-illumination-microscopy-3).
 
Jiaming Qian
added a research item
Since a slight variance in production processes can make the entire production run defective, defect inspections are indispensable procedures in manufacturing processes to ensure high quality of each item before entering the next manufacturing step. Three-dimensional (3D) optical shape measurement technologies are widely applied for surface defect inspection of complex workpieces because of its high-accuracy and digitization. However, the complex surface structure and position of the test object can pose serious challenges, making inspections still relatively slow, expensive, and complicated in implementation and maintenance. In this work, we propose a real-time 360∘ 3D surface defect inspection approach based on fringe projection profilometry without any auxiliary equipment for position control. Firstly, a multi-view 3D measurement based on geometric constraints is employed to acquire high-accuracy depth information from different perspectives. Then, a cycle-positioning-based registration scheme with the establishment of the pose-information-matched 3D standard digital model is proposed to realize rapid alignment of the measured point cloud and the standard model. Finally, a minimum 3D distance search is driven by a dual-thread processing mechanism for simultaneous scanning and detecting to quantify and locate 3D surface defects in real time. Experimental results show that our method can accurately identify the surface defects of complicated objects in real time in an extremely simple (hand-held) manner, saving a lot of operational expenses on precision alignment and position-orientation adjustment. The proposed method holds tremendous potential for quality control in many facets of industry, such as product development, testing, and manufacturing.
Jiaming Qian
added 2 research items
Since a slight variance in production processes can make the entire production run defective, defect inspections are indispensable procedures in manufacturing processes to ensure high quality of each item before entering the next manufacturing step. Three-dimensional (3D) optical shape measurement technologies are widely applied for surface defect inspection of complex workpieces because of its high-accuracy and digitization. However, the complex surface structure and position of the test object can pose serious challenges, making inspections still relatively slow, expensive, and complicated in implementation and maintenance. In this work, we propose a real-time 360∘ 3D surface defect inspection approach based on fringe projection profilometry without any auxiliary equipment for position control. Firstly, a multi-view 3D measurement based on geometric constraints is employed to acquire high-accuracy depth information from different perspectives. Then, a cycle-positioning-based registration scheme with the establishment of the pose-information-matched 3D standard digital model is proposed to realize rapid alignment of the measured point cloud and the standard model. Finally, a minimum 3D distance search is driven by a dual-thread processing mechanism for simultaneous scanning and detecting to quantify and locate 3D surface defects in real time. Experimental results show that our method can accurately identify the surface defects of complicated objects in real time in an extremely simple (hand-held) manner, saving a lot of operational expenses on precision alignment and position-orientation adjustment. The proposed method holds tremendous potential for quality control in many facets of industry, such as product development, testing, and manufacturing.
Jiaming Qian
added an update
3D scanners developed by our research group.
 
Jiaming Qian
added an update
We propose a real-time 3D surface defect detection technology, which requires no assistance of any mechanical auxiliary devices and professional software. In addition, we develop a software system including real-time high-accuracy 3D data acquisition, real-time point cloud registration and real-time defect inspection functions to verify the feasibility of the proposed algorithm. Figure (a) shows the reconstructed 3D model of the standard object, and Fig. (b) is the detection result of the defective object. We use pseudo color to indicate the size of the defect, and users can visually observe the size and location of the defect according to the color change. The whole inspection process can be seen in Visualization 1.
 
Jiaming Qian
added an update
We test four continuously moving scenarios to demonstrate the superiority of our approach in the dynamic target measurement. The measurement results are shown in Fig. 1. It can be seen from the left two columns of Fig. 1 that the multi-frame imaging characteristics of the PS algorithm lead to obvious motion-induced artifacts in the reconstruction results when encountering moving objects. In addition, due to the sensitivity to phase errors, the results acquired by traditional stereo phase unwrapping (SPU) methods obviously perform worse. Because of the single-shot nature of our approach, the measurement can be performed uninterruptedly without being affected by motion artifacts for dynamic scenarios, as shown in the right most column of Fig. 1. For more details, please refer to [J. Qian, S. Feng, T. Tao, Y. Hu, Y. Li, Q. Chen, and C. Zuo, “Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement,” APL Photonics 5, 046105 (2020). ]
 
Jiaming Qian
added a research item
Recovering the high-resolution three-dimensional (3D) surface of an object from a single frame image has been the ultimate goal long pursued in fringe projection profilometry (FPP). The color fringe projection method is one of the technologies with the most potential towards such a goal due to its three-channel multiplexing properties. However, the associated color imbalance, crosstalk problems, and compromised coding strategy remain major obstacles to overcome. Inspired by recent successes of deep learning for FPP, we propose a single-shot absolute 3D shape measurement with deep-learning-based color FPP. Through “learning” on extensive data sets, the properly trained neural network can “predict” the high-resolution, motion-artifact-free, crosstalk-free absolute phase directly from one single color fringe image. Compared with the traditional approach, our method allows for more accurate phase retrieval and more robust phase unwrapping. Experimental results demonstrate that the proposed approach can provide high-accuracy single-frame absolute 3D shape measurement for complicated objects.
Jiaming Qian
added an update
In the first experiment, we arbitrarily rotate the David model and realize its 360° modeling. The registration results and scenes over different time periods are shown in Fig. 1. The reconstruction time of the entire model is 70 s. Figure 2 shows the results after registration, from which we can see that all 3D shapes of the David model are well aligned. This experiment demonstrates that our method can obtain a high-quality 360° 3D model of handheld objects in real time. For more details, please refer to [J. Qian, S. Feng, T. Tao, Y. Hu, K. Liu, S. Wu, Q. Chen, and C. Zuo, “High-resolution real-time 3603d model reconstruction of a handheld object withfringe projection profilometry,” Optics letters 44, 5751–5754 (2019). ]
 
Jiaming Qian
added an update
We apply our deep-learning-based approach to the dynamic 360-degree 3D model reconstruction. A metallic workpiece is continually rotated for one cycle on a mechanical stage and measured with our approach (note that there are no objects of such type in our training and verification set). The measured results from two different perspectives are shown in Figs. (a)-(d). Due to the single-shot nature of our approach, no synchronization between the camera and project is needed, and the measurement can be performed uninterruptedly without being affected by motion artifacts. We register all the independent measurements into an integral 3D model, which is shown in Figs. (e) and (f). This experiment demonstrates the potential of our method for rapid reverse engineering applications. For more details, please refer to[ J. Qian, S. Feng, Y. Li, T. Tao, Q. Chen, and C. Zuo, “Single-shot absolute 3D shape measurement with deep-learning-based color fringe projection profilometry,” Optics Letters 45, 1842-1845 (2020). ]
 
Jiaming Qian
added 7 research items
The digitization of the complete shape of real objects has essential applications in fields of intelligent manufacturing, industrial detection, and reverse modeling. In order to build the full geometric models of rigid objects, the object must be moved relative to the measurement system (or the scanner must be moved relative to the object) to obtain and integrate views of the object from all sides, which not only complicates the system configuration but makes the whole process time-consuming. In this Letter, we present a high-resolution real-time 360° three-dimensional (3D) model reconstruction method that allows one to rotate an object manually and see a continuously updated 3D model during the scanning process. A multi-view fringe projection profilometry system acquires high-precision depth information about a handheld object from different perspectives and, meanwhile, the multiple views are aligned and merged together in real time. Our system employs stereo phase unwrapping and an adaptive depth constraint that can unwrap the phase of dense fringe images robustly without increasing the number of captured patterns. We then develop an efficient coarse-to-fine registration strategy to match the 3D surface segments rapidly. Our experiments demonstrate that our method can reconstruct the high-precision complete 3D model of complex objects under arbitrary rotation without any instrument assist and expensive pre/post-processing.
Jiaming Qian
added an update
Jiaming Qian
added a project goal
To realize real-time high-accuracy 3D shape measurement for intelligent manufacturing.