This paper presents a novel approach for matching 2-D points between a video projector and a digital camera. Our method is motivated by camera–projector applications for which the projected image needs to be warped to prevent geometric distortion. Since the warping process often needs geometric information on the 3-D scene obtained from a triangulation, we propose a technique for matching points in the projector to points in the camera based on arbitrary video sequences. The novelty of our method lies in the fact that it does not require the use of pre-designed structured light patterns as is usually the case. The backbone of our application lies in a function that matches activity patterns instead of colors. This makes our method robust to pose, severe photometric and geometric distortions. It also does not require calibration of the color response curve of the camera–projector system. We present quantitative and qualitative results with synthetic and real-life examples, and compare the proposed method with the scale invariant feature transform (SIFT) method and with a state-of-the-art structured light technique. We show that our method performs almost as well as structured light methods and significantly outperforms SIFT when the contrast of the video captured by the camera is degraded.
All content in this area was uploaded by Marc-Antoine Drouin on May 26, 2015
Content may be subject to copyright.
A preview of the PDF is not available
... Distortion also appears when the 3D surface is wrongly oriented with respect to the projector or has a non-planar shape. One typical solution is to prewarp the projected image according to the 3D shape of the scene and the observer's viewing angle [6,19,23]. In this way, the geometric correction allows the viewer to see the projected image as if it had been projected on a flat screen perpendicular to the projector. ...
... Let us mention that basic camera-projector color-based matching technique are doomed to fail since images captured by the camera are heavily degraded by non-linear color distortion [6]. As for feature-based matching strategies (using SIFT [16] for example), they do not produce a dense map which is essential in our case. ...
... As for feature-based matching strategies (using SIFT [16] for example), they do not produce a dense map which is essential in our case. Drouin et al. [6] proposed a camera-projector system that Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/pr uses activity patterns to find sparse matches. ...
This paper presents a camera-projector system which allows for 3D reconstruction and copyright protection. Our approach relies on video-embedded binary patterns whose proportion of black and white at each pixel correspond to the original grayscale value. When projected at a sufficiently high frame rate, these binary patterns are seamless to the observer and look as if a normal video was projected. These patterns have been designed to encode pixel position based on a collection of on-off/off-on transitions. When the camera is properly adjusted, these temporal transitions can be detected with a change detection method and converted into a 3D surface. However, when filmed by an uncalibrated camera, these transitions are not only unreadable, they induce spatial parasitic patterns in the recorded video.Our method is motivated by two families of applications. First, all applications for which a creative visual work needs to be pre-warped according to a 3D model to prevent geometric distortion when displayed on a dynamic scene. Typical examples include augmented reality and plays involving artistic staging. Second, all applications for which a multimedia copyrighted document (e.g. a movie projected in a theater) shall not be copied integrally with a hand-held digital video camera. The interference between the on-off/off-on transitions and the acquisition rate of most consumer-grade cameras creates disturbing visual artifacts. The system can even be adjusted to make sure a ‘VOID’ pattern appears in the recorded video.In this paper, we present how video frames are encoded and decoded and how the detection (and the nondetection) of temporal transitions allows for 3D reconstruction and copyright protection at the same time. Experimental results reveal that our system computes dense 3D maps at a rate of 11.25 fps and with an accuracy of approximately ±1/2 pixel, i.e. 56 microns when in focus and using a standoff distance of 75 cm.
... We used the similarity proposed by the binary motionbarcode descriptor [7] as one of the cues for our algorithm. Similar motion-based binary descriptors have been proposed and used by [15], [14]. Both methods assume a planar structure of the scene. ...
We address the problem of epipolar geometry using the motion of silhouettes. Such methods match epipolar lines or frontier points across views, which are then used as the set of putative correspondences. We introduce an approach that improves by two orders of magnitude the performance over state-of-the-art methods, by significantly reducing the number of outliers in the putative matching. We model the frontier points' correspondence problem as constrained flow optimization, requiring small differences between their coordinates over consecutive frames. Our approach is formulated as a Linear Integer Program and we show that due to the nature of our problem, it can be solved efficiently in an iterative manner. Our method was validated on four standard datasets providing accurate calibrations across very different viewpoints.
A model to predict and characterize the impact of out-of-focus blurring on the range uncertainty associated with the measurement by a phase-shift 3D scanner is presented. First, the reduction of the sine wave magnitude introduced by the projector lenses and the camera lenses is considered. Then, the noise reduction effects related to the camera image blurring introduced by the camera lenses are also included in the model. The main results of this study indicate that the uncertainty for a high-resolution system varies and exhibits a slanted “W” shape, which significantly differs from the inverse square of the range expected from the triangulation equation or the slanted “U” shape, which may be intuitively expected when combining blurring caused by a limited depth of field and the triangulation equation. We provide a comprehensive experimental characterization of a purposely constructed 3D scanner designed to isolate the performance degradation caused by out-of-focus projection and acquisition lenses. This scanner is designed to mimic the characteristics of a high-resolution scanner that can be employed for demanding quality control applications. In the tested configurations, the predicted depth-of-fields were within 16.3% of the corresponding measured values. This concordance between the theoretical results and experimental results suggests that the proposed model can be used to assist the design of phase-shift scanners.
Standard camera and projector calibration techniques use a checkerboard that is manually shown at different poses to determine the calibration parameters. Furthermore, when image geometric correction must be performed on a three-dimensional (3D) surface, such as projection mapping, the surface geometry must be determined. Camera calibration and 3D surface estimation can be costly, error prone, and time-consuming when performed manually. To address this issue, we use an auto-calibration technique that projects a series of Gray code structured light patterns. These patterns are captured by the camera to build a dense pixel correspondence between the projector and camera, which are used to calibrate the stereo system using an objective function, which embeds the calibration parameters together with the undistorted points. Minimization is carried out by a greedy algorithm that minimizes the cost at each iteration with respect to both calibration parameters and noisy image points. We test the auto-calibration on different scenes and show that the results closely match a manual calibration of the system. We show that this technique can be used to build a 3D model of the scene, which in turn with the dense pixel correspondence can be used for geometric screen correction on any arbitrary surface.
This paper presents a fast continuous geometric calibration method for projector-camera system under ambient light. Our method estimates an appropriate exposure time to prevent features in captured image from degradation and adopts ORB descriptor to match features pairs in real-time. The adaptive exposure method has been verified with different exposure values and proved to be effective. We also implement our real-time continuous calibration method on Dual-projection display. The calibration process can be accomplished smoothly within 5 frames.
Projectors are currently undergoing a transformation as they evolve from static output devices to portable, environment-aware, communicating systems. An enhanced projector can determine and respond to the geometry of the display surface, and can be used in an ad-hoc cluster to create a self-configuring display. Information display is such a prevailing part of everyday life that new and more flexible ways to present data are likely to have significant impact. This paper examines geometrical issues for enhanced projectors, relating to customized projection for different shapes of display surface, object augmentation, and co-operation between multiple units.We introduce a new technique for adaptive projection on nonplanar surfaces using conformal texture mapping. We describe object augmentation with a hand-held projector, including interaction techniques. We describe the concept of a display created by an ad-hoc cluster of heterogeneous enhanced projectors, with a new global alignment scheme, and new parametric image transfer methods for quadric surfaces, to make a seamless projection. The work is illustrated by several prototypes and applications.
A important problem is that of finding a quadric surface which gives a "best" fit to m given data points. There are many application areas, for example metrology, computer graphics, pattern recognition, and in particular quadric surfaces are often to be found in manufactured parts. There are many criteria which can be used for fitting, but one of the simplest is so-called algebraic fitting, which exploits the fact that an expression for the curve can be given which is affine in the free parameters. Here we examine a general class of such algebraic fitting problems, consider how the members of the class can be interpreted in terms of the errors in the data, and present simple algorithms which apply to all of the problems.
Most existing calibration techniques for multi-projector display system require that the display configuration remain fixed during the display process. We in this paper present a new approach to continuously re-calibrate the projection system to automatically adapt to the display configuration changes, while the multi-projector system is being used without interruption. By rigidly attaching a camera to each projector, we argument the projector with sensing capability and use the camera to provide online close-loop control. In contrast to previous auto or continuous projector calibration solutions, our approach can be used on surfaces of arbitrary geometry and can handle both projector and display surface movement, yielding more flexible system configuration and better scalability. Experimental results show that our approach is both accurate and robust.
Stereo matching is one of the most active research areas in computer vision. While a large number of algorithms for stereo correspondence have been developed, relatively little work has been done on characterizing their performance. In this paper, we present a taxonomy of dense, two-frame stereo methods. Our taxonomy is designed to assess the different components and design decisions made in individual stereo algorithms. Using this taxonomy, we compare existing stereo methods and present experiments evaluating the performance of many different variants. In order to establish a common software platform and a collection of data sets for easy evaluation, we have designed a stand-alone, flexible C++ implementation that enables the evaluation of individual components and that can easily be extended to include new algorithms. We have also produced several new multi-frame stereo data sets with ground truth and are making both the code and data sets available on the Web. Finally, we include a comparative evaluation of a large set of today's best-performing stereo algorithms.