[show abstract][hide abstract] ABSTRACT: Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.
IEEE Transactions on Pattern Analysis and Machine Intelligence 03/2012; 34(3):548-563. · 4.80 Impact Factor
[show abstract][hide abstract] ABSTRACT: Phase shifting structured light illumination for range sensing involves projecting a set of grating patterns where accuracy is determined, in part, by the number of stripes. However, high pattern frequencies introduce ambiguities during phase unwrapping. This paper proposes a process for embedding a period cue into the projected pattern set without reducing the signal-to-noise ratio. As a result, each period of the high frequency signal can be identified. The proposed method can unwrap high frequency phase and achieve high measurement precision without increasing the pattern number. Therefore, the proposed method can significantly benefit real-time applications. The method is verified by theoretical and experimental analysis using prototype system built to achieve 120 fps at 640 × 480 resolution.
IEEE Transactions on Image Processing 12/2011; · 3.20 Impact Factor
[show abstract][hide abstract] ABSTRACT: The use of structured light illumination techniques for three-dimensional data acquisition is, in many cases, limited to stationary subjects due to the multiple pattern projections needed for depth analysis. Traditional Composite Pattern (CP) multiplexing utilizes sinusoidal modulation of individual projection patterns to allow numerous patterns to be combined into a single image. However, due to demodulation artifacts, it is often difficult to accurately recover the subject surface contour information. On the other hand, if one were to project an image consisting of many thin, identical stripes onto the surface, one could, by isolating each stripe center, recreate a very accurate representation of surface contour. But in this case, recovery of depth information via triangulation would be quite difficult. The method described herein, Modified Composite Pattern (MCP), is a conjunction of these two concepts. Combining a traditional Composite Pattern multiplexed projection image with a pattern of thin stripes allows for accurate surface representation combined with nonambiguous identification of projection pattern elements. In this way, it is possible to recover surface depth characteristics using only a single structured light projection. The technique described utilizes a binary structured light projection sequence (consisting of four unique images) modulated according to Composite Pattern methodology. A stripe pattern overlay is then applied to the pattern. Upon projection and imaging of the subject surface, the stripe pattern is isolated, and the composite pattern information demodulated and recovered, allowing for 3D surface representation. Additionally, we introduce techniques which, when implemented, allow fully automated processing of the Modified Composite Pattern image.
[show abstract][hide abstract] ABSTRACT: Many applications require detection of multiple features that locally remain consistent in shape and intensity characteristics, but may globally change position with respect to one another over time or under different circumstances. We refer to these feature sets, defined by their characteristic relative positioning, as multifeature constellations. We introduce a method of processing in which multiple levels of correlation, using specially designed composite feature detection filters, are used to first detect local features, and then to detect constellations of these local features. We include experimental procedures and results indicating how the use of multifeature constellation detection may be utilized in applications such as sign language recognition and fingerprint matching.
[show abstract][hide abstract] ABSTRACT: There are applications that require detection of multiple features which remain consistent in shape locally, but may change position with respect to one another globally. We refer to these feature sets as multi-feature constellations. We introduce a multi-level correlation filter design which uses composite feature detection filters, which on one level detect local features, and then on the next level detect constellations of these local feature responses. We demonstrate the constellation filter method with sign language recognition and fingerprint matching.
[show abstract][hide abstract] ABSTRACT: To solve the problems associated with conventional 2-D fingerprint scanners such as skin deformation and print smearing, in this paper we introduce a noncontact fingerprint scanner employing structured light illumination to generate high-resolution albedo images as well as 3-D ridge scans. The question to be answered in this research is whether or not ridge depth information improves the quality and matching capability of acquired fingerprints? For evaluation of this question, we use the National Institute of Standards and Technology fingerprint image quality metrics. These metrics require the 3-D prints to be flattened. We present a complete and detailed flattening algorithm based upon unfolding an elastic tube fit to the peaks and valleys of ridges identified within the scan. Further improvement of the flattened print is achieved through the incorporation of ridge information extracted from the albedo image with the depth and albedo ridge information fused together according to local scan quality. Our study compares image quality between the flattened 3-D prints and ink rolled prints. Most significantly, the matching performance of 3-D flattened to 3-D flattened prints is evaluated based on ridge depth only, albedo only, and depth with albedo fusion.
IEEE Transactions on Information Forensics and Security 01/2011; · 1.90 Impact Factor
[show abstract][hide abstract] ABSTRACT: Structured light illumination by means of phase shifting patterns is a widely employed method for three-dimensional (3-D) image acquisition that is robust to ambient light and object albedo but may be especially susceptible to sensor and environment noise. In this paper, we study the specific technique of phase measuring profilometry (PMP) and the maximization of a pattern's signal to noise ratio (SNR). By treating the design of an N-pattern PMP process as placing points in an N-dimensional coding space, we define a pattern's SNR in terms of a pattern set's computational length and the number of coded phase periods in the projected patterns. Then, without introducing phase ambiguities, we propose a so-called edge pattern strategy that maximizes the computational length and number of periods. Theoretically, the edge pattern technique improves the SNR by 1.2381 times when using three component patterns and by 15.5421 times when using five patterns. Experimental results further demonstrate the improved SNR of the proposed edge pattern technique such that more accurate 3-D results are achieved using fewer component patterns.
Journal of the Optical Society of America A 09/2010; 27(9):1962-71. · 1.67 Impact Factor
[show abstract][hide abstract] ABSTRACT: Structured-light illumination is a process of three-dimensional imaging where a series of time-multiplexed, striped patterns are projected onto a target scene with the corresponding captured images used to determine surface shape according to the warping of the projected patterns around the target. In a real-time system, a high-speed projector/camera pair is used such that any surface motion is small over the projected pattern sequence, but regardless of acquisition speed, there are always those pixels near the edge of a moving surface that capture the projected patterns on both fore- and background surfaces. These edge pixels then create unpredictable results that typically require expensive processing steps to remove, but in this Letter, we introduce a filtering process that identifies motion artifacts based upon the discrete Fourier transform applied to the time axis of the captured pattern sequence. The process is of very low computational complexity, and in this Letter, we demonstrate that in a real-time structured-light illumination (SLI) system, the process comes at a cost of 15 frames per second (fps), where our SLI system drops from 180 to 165 fps after deleting those edge pixels where motion was detected.
[show abstract][hide abstract] ABSTRACT: Phase measuring profilometry is a method of structured light illumination whose three-dimensional reconstructions are susceptible to error from nonunitary gamma in the associated optical devices. While the effects of this distortion diminish with an increasing number of employed phase-shifted patterns, gamma distortion may be unavoidable in real-time systems where the number of projected patterns is limited by the presence of target motion. A mathematical model is developed for predicting the effects of nonunitary gamma on phase measuring profilometry, while also introducing an accurate gamma calibration method and two strategies for minimizing gamma's effect on phase determination. These phase correction strategies include phase corrections with and without gamma calibration. With the reduction in noise, for three-step phase measuring profilometry, analysis of the root mean squared error of the corrected phase will show a 60x reduction in phase error when the proposed gamma calibration is performed versus 33x reduction without calibration.
Journal of the Optical Society of America A 03/2010; 27(3):553-62. · 1.67 Impact Factor
[show abstract][hide abstract] ABSTRACT: A novel dual-frequency pattern is developed which combines a high-frequency sinusoid component with a unit-frequency sinusoid component, where the high-frequency component is used to generate robust phase information, and the unit-frequency component is used to reduce phase unwrapping ambiguities. With our proposed pattern scheme, phase unwrapping can overcome the major shortcomings of conventional spatial phase unwrapping: phase jumping and discontinuities. Compared with conventional temporal phase unwrapped approaches, the proposed pattern scheme can achieve higher quality phase data using a less number of patterns. To process data in real time, we also propose and develop look-up table based fast and accurate algorithms for phase generation and 3-D reconstruction. Those fast algorithms can be applied to our pattern scheme as well as traditional phase measuring profilometry. For a 640 x 480 video stream, we can generate phase data at 1063.8 frames per second and full 3-D coordinate point clouds at 8.3 frames per second. These achievements are 25 and 10 times faster than previously reported studies.
[show abstract][hide abstract] ABSTRACT: To solve problems associated with conventional 2D fingerprint acquisition processes including skin deformations and print smearing, we developed a noncontact 3D fingerprint scanner employing structured light illumination that, in order to be backwards compatible with existing 2D fingerprint recognition systems, requires a method of unwrapping the 3D scans into 2D equivalent prints. For the latter purpose of virtually flattening a 3D print, this paper introduces a fit-sphere unwrapping algorithm. Taking advantage of detailed 3D information, the proposed method defuses the unwrapping distortion by controlling the distances between neighboring points. Experimental results will demonstrate the high quality and recognition performance of the 3D unwrapped prints versus traditionally collected 2D prints. Furthermore, by classifying the 3D database into high- and low-quality data sets, we demonstrate that the relationship between quality and recognition performance holding for conventional 2D prints is achieved for 3D unwrapped fingerprints.
[show abstract][hide abstract] ABSTRACT: The use of fingerprints as a biometric is both the oldest mode of computer-aided personal identification and the most-relied-on technology in use today. However, current acquisition methods have some challenging and peculiar difficulties. For higher performance fingerprint data acquisition and verification, a novel noncontact 3-D fingerprint scanner is investigated, where both the detailed 3-D and albedo information of the finger is obtained. The obtained high-resolution 3-D prints are further converted into 3-D unraveled prints, to be compatible with traditional 2-D automatic fingerprint identification systems. As a result, many limitations imposed on conventional fingerprint capture and processing can be reduced by the unobtrusiveness of this approach and the extra depth information acquired. To compare the quality and matching performances of 3-D unraveled with traditional 2-D plain fingerprints, we collect both 3-D prints and their 2-D plain counterparts. The print quality and matching performances are evaluated and analyzed by using National Institute of Standard Technology fingerprint software. Experimental results show that the 3-D unraveled print outperforms the 2-D print in both quality and matching performances.
[show abstract][hide abstract] ABSTRACT: For many years, fingerprints have been captured by pressing a finger against a paper or hard surface. This touch-based fingerprint acquisition introduces some problems such as distortions and deformations in the acquired images, which arise due to the contact of the fingerprint surface with the sensor platen, and degrades the recognition performance. A new touch-less fingerprint technology has been recently introduced to the market, which can address the problems with the contact-based fingerprint systems. In this paper, we propose a new algorithm for unwrapping the acquired 3-D scan of the subject's finger into a 2-D rolled equivalent image. Therefore, The resulting image can be matched with the conventional 2-D scans; it also can be used for matching unwrapped 3-D fingerprints among themselves with the 2-D fingerprint matching algorithms. The algorithm is based on curvature analysis of the 3-D surface. The quality of the resulting image is evaluated and analyzed using NIST fingerprint image software.
Biometrics: Theory, Applications, and Systems, 2009. BTAS '09. IEEE 3rd International Conference on; 10/2009
[show abstract][hide abstract] ABSTRACT: This paper introduces a new technology of non-contact 3D fingerprint capture and processing for higher quality fingerprint data acquisition. The system relies on a real-time 3D sensor using structured light illumination (SLI), which generates both texture and detailed ridge depth information. The high resolution 3D scans are then converted into flat equivalent images. As a result, many limitations imposed upon conventional fingerprint capture and processing can be reduced by the unobtrusiveness of this approach and the extra depth information acquired. The image quality is evaluated and analyzed using NIST fingerprint image software. A comparison is performed between the unraveled 3D fingerprints and their 2D plain counterparts in terms of fingerprint quality.
Biometrics, Identity and Security (BIdS), 2009 International Conference on; 10/2009
[show abstract][hide abstract] ABSTRACT: Our group has developed several methods for acquiring 3-dimensional objects in motion which include facial expressions. For this to be practical we need to identify and track various features contained in facial expressions. To accomplish this we introduce a set of feature based trackers and propose strategies for combining them together to form meshes. We present our strategy in the context of swarm theory where the elements of the swarm are the feature trackers and the communication structure of the swarm is essentially a spatial mesh. We demonstrate the concepts with examples of facial feature tracking.
[show abstract][hide abstract] ABSTRACT: Human to computer interfaces have been so far dominated by hand held and/or physical interfaces such as keyboards, mice, joysticks, touch screens, light pens, etc.. There has been considerable study in the use of non-contact interface technology that use image motion, stereo vision, and time of flight ranging devices. Using image processing of a single camera image, there is difficulty segmenting the feature of interest and poor depth accuracy. Stereo vision requires two cameras and is dependent on distinct features on the surface/object being measured, and time of flight systems are very expensive and lack close range accuracy. We believe that Structured Light Illumination is a practical solution to the non-contact interface problem because of the simplicity of one camera and one projector, and its direct and accurate measurement of human hands and faces. Furthermore, with the advent of projected keyboards for augmented reality interfacing, a camera and projector are already present. In fact, the keyboard pattern could be used as the SLI pattern. In general, SLI, particularly the single pattern methods described in this research, are accurate, surface feature independent, and require only a simple slide projection in either visible or NearInfra-Red light frequencies. The illumination source only requires efficient LED based illumination technology. As discussed in the results section, the accuracy of the depth measurement is within 1 mm so the demonstration is not just a non-contact "mouse" but a five finger analog controller. Full finger motion control could be used for a wide range of
[show abstract][hide abstract] ABSTRACT: We present an eight million point structured light illumination scanner
design. It has a single patch projection resolution of 12,288 lines
along the phase direction. The Basler CMOS video cameras are 2352 by
1726 pixel resolution. The configuration consists of a custom Boulder
Nonlinear Systems Spatial Light Modulator for the projection system and
dual four mega pixel digital video cameras. The camera field of views
are tiled with minimal overlap region and a potential capture rate of 24
frames per second. This report is a status report of a project still
under development. We will report on the concept of applying a 1D-square
footprint projection chip and give preliminary results of single camera
scans. The structured light illumination technique we use is the
multi-pattern, multi-frequency phase measuring profilometry technique
already published by our group.
[show abstract][hide abstract] ABSTRACT: Fingerprints are one of the most commonly used and relied-upon biometric technology. But often the captured fingerprint image is far from ideal due to imperfect acquisition techniques that can be slow and cumbersome to use without providing complete fingerprint information. Most of the diffculties arise due to the contact of the fingerprint surface with the sensor platen. To overcome these diffculties we have been developing a noncontact scanning system for acquiring a 3-D scan of a finger with suffciently high resolution which is then converted into a 2-D rolled equivalent image. In this paper, we describe certain quantitative measures evaluating scanner performance. Specifically, we use some image software components developed by the National Institute of Standards and Technology, to derive our performance metrics. Out of the eleven identified metrics, three were found to be most suitable for evaluating scanner performance. A comparison is also made between 2D fingerprint images obtained by the traditional means and the 2D images obtained after unrolling the 3D scans and the quality of the acquired scans is quantified using the metrics.