Alexander Neubeck’s research while affiliated with ETH Zurich and other places

This paper investigates several aspects of 3D-2D camera pose estimation, aimed at robot navigation in poorly-textured scenes. The major contribution is a fast, linear algorithm for the general case with six or more points. We show how to specialise this to work with only four or five points, which is of utmost importance in a test and hypothesis framework. Our formulation allows for an easy inclusion of lines, as well as the handling of other cam- era geometries, such as stereo rigs. We also treat the special case of planar motion, a valid restriction for most indoor environments. We conclude the paper with extensive simulated tests and a real test case, which substantiate the algorithm's usability for our application domain.

This paper addresses the problem of camera self-calibration, bundle adjustment and 3D reconstruction from line segments in two images of poorly-textured indoor scenes. First, we generate line segment correspondences, using an extended version of our previously proposed matching scheme. The first main contribution is a new method to identify polyhedral junctions resulting from the intersections of the line segments. At the same time, the images are segmented into planar polygons. This is done using an algorithm based on a binary space partitioning (BSP) tree. The junctions are matched end points of the detected line segments and hence can be used to obtain the epipolar geometry. The essential matrix is considered for metric camera calibration. For better stability, the second main contribution consists in a bundle adjustment on the line segments and the camera parameters that reduces the number of unknowns by a maximum flow algorithm. Finally, a piecewise-planar 3D reconstruction is computed based on the segmentation of the BSP tree. The system's performance is tested on some challenging examples.

Working with IBR/BTF data requires a complete calibration. Modern setups speed up recordings by using multiple lamps and cameras. Therefore, the calibration task gets more time consuming and challenging, especially for an accurate calibration of multiple light sources. Most works are dedicated to the calibration of cameras, whereas the light field calibration problem remains as a rule overlooked. Our experiments have shown that the spatial variance of light strength can be vigorous, inducing serious damage to the IBR/BTF data. We propose a novel method based on Helmholtz reciprocity, which derives light field information directly from the target IBR/BTF data rather than from specially dedicated calibration objects. Instead of repeating the recording of a huge number of images, only one additional image is needed.

In this work we scrutinize a low level computer vision task - non-maximum suppression (NMS) - which is a crucial preprocessing step in many computer vision applications. Especially in real time scenarios, efficient algorithms for such preprocessing algorithms, which operate on the full image resolution, are important. In the case of NMS, it seems that merely the straightforward implementation or slight improvements are known. We show that these are far from being optimal, and derive several algorithms ranging from easy-to-implement to highly-efficient

Textures often result from complicated surface geome-tries. We propose a method that extracts such geometry from raw BTF data. Exploiting the huge amount of input data, only a few and rather weak assumptions about reflectance and geometry suffice. A key element of our approach are viewpoint robustness of reflectance features. We propose a few and compare them for 3D reconstruction.

We present a GPU-based foreground-background segmentation that processes image sequences in less than 4ms per frame. Change detection wrt. the background is based on a color similarity test in a small pixel neighbourhood, and is integrated into a Bayesian estimation framework. An iterative MRF-based model is applied, exploiting parallelism on modern graphics hardware. Resulting segmentation exhibits compactness and smoothness in foreground areas as well as for inter-frame temporal contigu-ity. Further refinements extend the colinearity cri-terion with compensation for dark foreground and background areas and thus improving overall per-formance.

The purpose of this work is to synthesize textures of rough, real world surfaces under freely chosen viewing and illumination directions. Moreover, such textures are produced for continuously changing directions in such a way that the different textures are mutually consistent, i.e. emulate the same piece of surface. This is necessary for 3D animation. It is assumed that the mesostructure (small-scale) geometry of a surface is not known, and that the only input consists of a set of images, taken under different viewing and illumination directions. These are automatically aligned to build an appropriate bidirectional texture function (BTF). Directly extending 2D synthesis methods for pixels to complete BTF columns has drawbacks which are exposed, and a superior sequential but highly parallelizable algorithm is proposed. Examples demonstrate the quality of the results.

Industrial augmented reality (AR) applications require fast, robust, and precise tracking. In environments where conventional high-end tracking systems cannot be applied for certain reasons, marker-based tracking can be used with success as a substitute if care is taken about (1) calibration and (2) run-time tracking fidelity. In out-of-the-laboratory environments multi-marker tracking is needed because the pose estimated from a single marker is not stable enough. The overall pose estimation can be dramatically improved by fusing information from several markers fixed relative to each other compared to a single marker only. To achieve results applicable in an industrial context relative marker poses need to be properly calibrated. We propose a semiautomatic image-based calibration method requiring only minimal interaction within the workflow. Our method can be used off-line, or preferably incrementally online. When used online, our method shows reasonably good accuracy and convergence with workflow interruption of less than one second per incremental step. Thus, it can be interactively used. We illustrate our method with an industrial application scenario.

Texture synthesis through so-called 'smart copying' requires a seamless meshing of texture subparts. Current systems first select subparts that seem to globally fit well and then optimise the cut between them on a rather local scale. This order of first selecting patches and then caring about the seamless meshing reduces one's leeway in the choice of seamless cuts to a small zone of overlap between the patches. Therefore, even in the latest and smartest of smart copying approaches seams still tend to show up in the results. Here we present an approach that first looks for promising cuts, and uses these as the point of departure. It is shown that even a simple criterion for the quality of seams already supports high-quality smart copying and texture tessellation.