No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Modelling fringe projection based on linear systems theory and geometric transformation
Abstract
Fringe projection measurement techniques offer fast, non-contact measurements of the surface form of manufactured parts at relatively low cost. Recent advances in fringe projection have reduced measurement errors from effects such as multiple surface reflections and projector defocus. However, there is no standardised calibration framework for fringe projection systems and an uncertainty estimation of surface measurements is rarely carried out in practice. A calibration framework for estimating spatial frequency-dependent measurement uncertainty built on solid theoretical foundations is required. To move towards traceable surface measurement using fringe projection techniques, we are developing a measurement model to accurately predict the captured image and include all major uncertainty contributors, i.e. a virtual fringe projection system. The first step of the model is to calculate the optical field distribution using the three-dimensional optical transfer function of the projector. Next, a camera image is built up using a ray-tracing model to probe the optical field distribution at the measurement surface boundary. The results are compared to an experimental fringe projection system. The intention is to use this model within a Monte-Carlo framework to move towards estimating the uncertainty at each point-cloud data point.
... In 2019, Pooya Ghandali et al. [43] proposed a structured light technique for performance assessment in dimensional and topographical measurements. In the same year, George Gayton et al. [44] developed In multi-eye stereo vision measurements, the structured light 3D measurement method is the most commonly applied method and has been widely used in plant surface feature recognition, human body feature recognition, 3D sensing, cultural relic protection, metal sheet size control and medical fields [35][36][37][38][39][40][41][42]. How to effectively evaluate the measurement stability of many structured light 3D measuring instruments is one of the keys to improving the processing quality of manufacturing products. ...
... In 2019, Pooya Ghandali et al. [43] proposed a structured light technique for performance assessment in dimensional and topographical measurements. In the same year, George Gayton et al. [44] developed an evaluation framework for the uncertainty of the extraction of feature point clouds on the surface of objects to correct them in time. ...
The three-dimensional (3D) size and morphology of high-temperature metal components need to be measured in real time during manufacturing processes, such as forging and rolling. Since the surface temperature of a metal component is very high during the forming and manufacturing process, manually measuring the size of a metal component at a close distance is difficult; hence, a non-contact measurement technology is required to complete the measurement. Recently, machine vision technology has been developed, which is a non-contact measurement technology that only needs to capture multiple images of a measured object to obtain the 3D size and morphology information, and this technology can be used in some extreme conditions. Machine vision technology has been widely used in industrial, agricultural, military and other fields, especially fields involving various high-temperature metal components. This paper provides a comprehensive review of the application of machine vision technology in measuring the 3D size and morphology of high-temperature metal components. Furthermore, according to the principle and method of measuring equipment structures, this review highlights two aspects in detail: laser scanning measurement and multi-view stereo vision technology. Special attention is paid to each method through comparisons and analyses to provide essential technical references for subsequent researchers.
... In addition, vision-based 3D shape measurement methods face difficulties for surfaces with specular reflection, transparency and high dynamic range. Though researchers have presented various strategies [150][151][152][153][154][155], they are not robust enough for arbitrary scenes, and their consistency and repeatability are often difficult to guarantee. Recently, great advancements in AI technology have facilitated the development of vision-based 3D shape measurement techniques and great progress has been made in image processing using deep learning-based networks instead of conventional methods [156]. ...
Vision-based three-dimensional (3D) shape measurement techniques have been widely applied over the past decades in numerous applications due to their characteristics of high precision, high efficiency and non-contact. Recently, great advances in computing devices and artificial intelligence have facilitated the development of vision-based measurement technology. This paper mainly focuses on state-of-the-art vision-based methods that can perform 3D shape measurement with high precision and high resolution. Specifically, the basic principles and typical techniques of triangulation-based measurement methods as well as their advantages and limitations are elaborated, and the learning-based techniques used for 3D vision measurement are enumerated. Finally, the advances of, and the prospects for, further improvement of vision-based 3D shape measurement techniques are proposed.
In this paper, we will review the development and use of an ISO standardised framework to allow calibration of surface topography measuring instruments. We will draw on previous work to present the state of the art in the field in terms of employed methods for calibration and uncertainty estimation based on a fixed set of metrological characteristics. The resulting standards will define the metrological characteristics and present default methods and material measures for their determination—the paper will summarise this work and point out areas where there is still some work to do. An example uncertainty estimation is given for an optical topography measuring instrument, where the effect of topography fidelity is considered.
Large-scale surfaces are prevalent in advanced manufacturing industries, and 3D profilometry of these surfaces plays a pivotal role for quality control. This paper proposes a novel and flexible large-scale 3D scanning system assembled by combining a robot, a binocular structured light scanner and a laser tracker. The measurement principle and system construction of the integrated system are introduced. A mathematical model is established for the global data fusion. Subsequently, a robust method is introduced for the establishment of the end coordinate system. As for hand-eye calibration, the calibration ball is observed by the scanner and the laser tracker simultaneously. With this data, the hand-eye relationship is solved, and then an algorithm is built to get the transformation matrix between the end coordinate system and the world coordinate system. A validation experiment is designed to verify the proposed algorithms. Firstly, a hand-eye calibration experiment is implemented and the computation of the transformation matrix is done. Then a car body rear is measured 22 times in order to verify the global data fusion algorithm. The 3D shape of the rear is reconstructed successfully. To evaluate the precision of the proposed method, a metric tool is built and the results are presented.
This article examines the influence of incident angle, object colour and measurement distance on the computer numerically
controlled laser scanning process. To determine the physical background of these influences, the operation of the triangulation
sensor, the surface reflection and the colour properties of the measured object were analysed. The various influences and
their physical background are explained by using a test-measurement setup, which makes it possible to investigate a specific
influencing factor. The article concludes with several guidelines that should be followed in order to obtain better measurement
results.
KeywordsLaser scanner-Accuracy-3D scanning-Surface acquisition-Point cloud-Reverse engineering
High-speed 3D shape measurement (or imaging) has seen tremendous growths over the past decades, especially the past few years due to the improved speed of computing devices and reduced costs of hardware components. 3D shape measurement technologies have started penetrating more into our daily lives than ever before with the recent release of iPhone X that has an built-in 3D sensor for Face ID, along with prior commercial success of inexpensive commercial sensors (e.g., Microsoft Kinect). This paper overviews the primary state-of-the-art 3D shape measurement techniques based on structured light methods, especially those that could achieve high measurement speed and accuracy. The fundamental principles behind those technologies will be elucidated, experimental results will be presented to demonstrate capabilities and/or limitations for those popular techniques, and finally present our perspectives on those remaining challenges to be conquered to make advanced 3D shape measurement techniques ubiquitous.
Anti-spoof touchless imaging of human ear has strong potential to be a useful alternative to traditional biometric sensors. The features in a human ear are more reliable than other physical traits like fingerprint, face, palm print, etc. In this paper, we propose an efficient ear pattern sensor by introducing a new imaging technique that is capable of simultaneous retrieval of the topographic details as well as its liveliness information. In the proposed technique, a collimated laser beam is made incident on a sinusoidal grating, and the resultant structured pattern is projected towards the ear specimen. For detection of topographic information and liveliness, time series bio-speckled fringe sequences are recorded using a CCD camera. For noise removal and shape reconstruction from the acquired information, a combination of windowed Fourier filtering and low pass filtering is used. Discrete cosine transform based fast biospeckle algorithm is introduced for assessment of genuineness (i.e. to create distinction between real and fake samples) of the specimen. The proposed sensor is simple, compact, full-field and robust against different spoof attacks.
Blade with complex and thin-walled shape is one of the critical parts in an aircraft engine, and its whole 3D shape should be measured to ensure the efficiency and safety of the engines. But there hardly exist an omnipotent blade 3D measurement method that can simultaneously address accuracy, efficiency and ability of measuring high-reflective surface. To this end, a hybrid 3D metrology method integrated by fringe projection profilomety (FPP) and conoscopic holography (CH) is developed, and FPP is responsible for gaining measurement accuracy and efficiency, while CH scans the high-reflective areas that FPP fails to measure. In this integrated system, the key task involved is calibration of rigid transformation between coordinate systems of inhomogeneous 3D sensors, so an accurate calibration method is proposed. It only requires FPP sensor and CH sensor to measure a planar target at several different poses in the measurement volume, and then the measured space planes can be used to compute rigid transformation. In order to enhance its robustness to noise, a non-linear optimization model that considers signal-to-noise ratio (SNR) of CH sensor is establish for obtaining the optimal calibration results. The simulations and real experiments demonstrate that proposed method can achieve accurate and robust calibration of inhomogeneous 3D sensors for measuring the blade.
We investigate the feasibility of using fringe projection to monitor the powder bed of a polyamide 12 polymer laser sintering machine. In particular, we demonstrate the ability of fringe projection to identify a number of defects arising during the printing process by recording the three-dimensional structure of the sintered powder bed after the completion of each layer. The defects identified ranged in size from hundreds of micrometres to hundreds of millimetres. The three-dimensional analysis of the powder bed data has shown the ability to quantify effects, such as curling, powder spreader blade interactions and the consolidation of a sintered layer. It has, therefore, been shown that the use of fringe projection in polymer laser sintering machines can provide deeper understanding and monitoring of the dynamic behaviour during the process. Fringe projection has shown potential to become part of a feedback and control system that interrupts the build and corrects for in-process defects where possible.
We introduce a flexible error correction method for fringe projection profilometry (FPP) systems in the presence of local blur phenomenon. Local blur caused by global light transport such as camera defocus, projector defocus, and subsurface scattering will cause significant systematic errors in FPP systems. Previous methods, which adopt high-frequency patterns to separate the direct and global components, fail when the global light phenomenon occurs locally. In this paper, the influence of local blur on phase quality is thoroughly analyzed, and a concise error correction method is proposed to compensate the phase errors. For defocus phenomenon, this method can be directly applied. With the aid of spatially varying point spread functions and local frontal plane assumption, experiments show that the proposed method can effectively alleviate the system errors and improve the final reconstruction accuracy in various scenes. For a subsurface scattering scenario, if the translucent object is dominated by multiple scattering, the proposed method can also be applied to correct systematic errors once the bidirectional scattering-surface reflectance distribution function of the object material is measured.
When fringe projection profilometry is used for measuring texture on rough surfaces, the measurement resolution is subject to the spatial frequency response of the instrument. The instrument transfer function (ITF) is a good metric to quantify this property. A valid ITF analysis requires the system to be linear. In this paper, we investigate the validity of using ITF to characterize the spatial resolution of a fringe projection system. Approximate linearity is shown through a mathematical analysis and simulation. We also demonstrate a practical method for measuring ITF using a stepped surface. The measured ITF is compared with an ITF prediction, which is simulated with a theoretical model.
A new fibre optic Lloyd’s mirror method is developed for extracting 3-D height distribution of various objects at the micron scale with a resolution of 4 µm. The fibre optic assembly is elegantly integrated to an optical microscope and a CCD camera. It is demonstrated that the proposed technique is quite suitable and practical to produce an interference pattern with an adjustable frequency. By increasing the distance between the fibre and the mirror with a micrometre stage in the Lloyd’s mirror assembly, the separation between the two bright fringes is lowered down to the micron scale without using any additional elements as part of the optical projection unit. A fibre optic cable, whose polymer jacket is partially stripped, and a microfluidic channel are used as test objects to extract their surface topographies. Point by point sensitivity of the method is found to be around 8 µm, changing a couple of microns depending on the fringe frequency and the measured height. A straightforward calibration procedure for the phase to height conversion is also introduced by making use of the vertical moving stage of the optical microscope. The phase analysis of the acquired image is carried out by One Dimensional Continuous Wavelet Transform for which the chosen wavelet is the Morlet wavelet and the carrier removal of the projected fringe patterns is achieved by reference subtraction. Furthermore, flexible multi-frequency property of the proposed method allows measuring discontinuous heights where there are phase ambiguities like 2π by lowering the fringe frequency and eliminating the phase ambiguity.
Coherence scanning interferometry (CSI) offers three dimensional (3D) measurement of surface topography with high precision and accuracy. Defocus within the interferometric objective lens, however, is commonly present in CSI measurements, and reduces both the resolving power of the imaging system and the ability to measure tilted surfaces. This paper extends the linear theory of CSI to consider the effects of defocus on the 3D transfer function and the point spread function in an otherwise ideal CSI instrument. The results are compared with measurements of these functions in a real instrument. This work provides further evidence for the validity of the linear systems theory of CSI.
Although coherence scanning interferometry (CSI) is capable of measuring surface form with sub-nanometre precision, it is well known that the performance of measuring instruments depends strongly on the local tilt and curvature of the sample surface. Based on 3D linear systems theory, however, a recent analysis of fringe generation in CSI provides a method to characterise the performance of surface measuring instruments and offers considerable insight into the origins of these errors. Furthermore, from the measurement of a precision sphere, a process to calibrate and partially correct instruments has been proposed. This paper presents, for the first time, a critical look at the calibration and correction process. Computational techniques are used to investigate the effects of radius error and measurement noise introduced during the calibration process for the measurement of spherical and sinusoidal profiles. Care is taken to illustrate the residual tilt and curvature dependent errors in a manner that will allow users to estimate measurement uncertainty. It is shown that by calibrating the instrument correctly and using appropriate methods to extract phase from the resulting fringes (such as frequency domain analysis), CSI is capable of measuring the profile of surfaces with varying tilt with sub-nanometre accuracy.
In transmission microscopy, many objects are three dimensional, that is, they are thicker than the depth of focus of the imaging system. The three-dimensional (3-D) image-intensity distribution consists of a series of two-dimensional images (optical slices) with different parts of the object in focus. First, we deal with the fundamental limitations of 3-D imaging with classical optical systems. Second, a transfer theory of 3-D image formation is derived that relates the 3-D object (complex index of refraction) to the 3-D image intensity distribution in first-order Born approximation. This theory applies to weak objects that do not scatter much light. Since, in a microscope, the illumination is neither coherent nor completely incoherent, a theory for partially coherent light is needed, but in this case the object phase distribution and the absorptive parts of the object play different roles. Finally, some experimental results are presented.
Global or indirect illumination effects such as interreflections and subsurface scattering severely degrade the performance of structured light-based 3D scanning. In this paper, we analyze the errors in structured light, caused by both long-range (interreflections) and short-range (subsurface scattering) indirect illumination. The errors depend on the frequency of the projected patterns, and the nature of indirect illumination. In particular, we show that long-range effects cause decoding errors for low-frequency patterns, whereas short-range effects affect high-frequency patterns.
Based on this analysis, we present a practical 3D scanning system which works in the presence of a broad range of indirect illumination. First, we design binary structured light patterns that are resilient to individual indirect illumination effects using simple logical operations and tools from combinatorial mathematics. Scenes exhibiting multiple phenomena are handled by combining results from a small ensemble of such patterns. This combination also allows detecting any residual errors that are corrected by acquiring a few additional images. Our methods can be readily incorporated into existing scanning systems without significant overhead in terms of capture time or hardware. We show results for several scenes with complex shape and material properties.
In this paper, we characterize 3D optical imaging techniques as 3D linear shift-invariant filtering operations. From the Helmholtz equation that is the basis of scalar diffraction theory, we show that the scattered field, or indeed a holographic reconstruction of this field, can be considered to be the result of a linear filtering operation applied to a source distribution. We note that if the scattering is weak, the source distribution is independent of the scattered field and a holographic reconstruction (or in fact any far-field optical imaging system) behaves as a 3D linear shift-invariant filter applied to the refractive index contrast (which effectively defines the object). We go on to consider tomographic techniques that synthesize images from recordings of the scattered field using different illumination conditions. In our analysis, we compare the 3D response of monochromatic optical tomography with the 3D imagery offered by confocal microscopy and scanning white light interferometry (using quasi-monochromatic illumination) and explain the circumstances under which these approaches are equivalent. Finally, we consider the 3D response of polychromatic optical tomography and in particular the response of spectral optical coherence tomography and scanning white light interferometry.
One of the main tasks of the quality test is the inspection of all relevant geometric parts related to the predefined tolerance range, whereas the uncertainty of measurement has to be less than the tolerance range. The reachable uncertainty of measurement can be determined using method A of the ISO Guide to the Expression of Uncertainty in Measurement (GUM), which is expensive and time consuming and has to be carried out for each individual metrologic case. Furthermore, it is possible to check the suitability of the measurement system for the planned inspection using virtual measurement techniques and therewith to reduce the time and money spent. This means that the uncertainty of measurement is estimated using method B of the GUM. In this paper, a virtual fringe projection system is used for the estimation of the uncertainty of measurement, which is compared with the uncertainty of measurement determined with a real measurement system using method A of the GUM. With the presented method, it is possible to calculate an optimal measurement position within the measurement volume, based on a minimum uncertainty of measurement. Thereby, the influence of the operator related to the uncertainty can be significantly reduced.
Fringe-projection measurement systems are susceptible to image saturation and measurement error if ambient illuminance is increased during measurement after earlier calibration. This paper presents analyses of fringe-projection measurement-accuracy sensitivity to object illuminance and fringe-pattern gray levels for an adaptive approach to fringe-pattern projection. Calibrations of a measurement system with no ambient light using eight different maximum fringe-pattern gray levels between 175 and 255 were performed utilizing the two-step triangular-pattern phase-shifting method. Measurements using the same patterns and corresponding calibration parameters performed with 0 and 300 lx ambient illuminance had mean RMS errors over depth below 0.5 and 0.7 mm, respectively, for maximum gray levels ranging from 195 to 255. The optimal tradeoff of lower maximum gray level to tolerate ambient lighting and avoid image saturation and higher maximum gray level for greater robustness to image noise occurred at 215, where the minimum error was 0.44 mm. In an adaptive fringe-pattern projection approach, the maximum gray level would be adjusted based on images acquired during measurement, and calibration parameters determined in earlier calibrations would be utilized. The approach is applicable to single or multiple-step measurement methods.
This paper aims at giving an overview on the state of the art of vision-based, non-contact 3D measurement systems based on structured light. Such systems are equipped with a camera and a light projector, and are referred to in the literature as active stereo systems. This report is structured into structured light, calibration methods and 3D measurement techniques. Moreover, several selected research works are described in order to illustrate these different topics
Geometrical product specifications (GPS) -Coordinate measuring machines (CMM): Technique for determining the uncertainty of measurement
ISO 15530: Geometrical product specifications (GPS) -Coordinate measuring machines (CMM):
Technique for determining the uncertainty of measurement, 2013