Conference Paper

An Image-Based Scene Representation and Rendering Framework Incorporating Multiple Representations.

Conference: Proceedings of the Vision, Modeling, and Visualization Conference 2003 (VMV 2003), München, Germany, November 19-21, 2003
Source: DBLP

ABSTRACT

A variety of image-based scene representations like light fields, concentric mosaics, panoramas, and omnidirectional video have been proposed in the past years. These image-based scene represen- tations provide photorealistic interactive user navigation in a 3D scene. As the trade-off between acquisition complexity, freedom of movement and rendering quality differs for the diverse tech- niques, the most efficient scene representation and rendering technique should be selected with re- spect to scene content and complexity. Splitting a scene into partial representations which are adapted to local requirements is pro- posed in this paper. Besides meaningful restric- tions to user movement, the transition between different image-based scene representations is addressed to provide an efficient image based walkthrough for large and complex scenes. We identify rendering parameters to achieve a seam- less transition between different representations and present results for stitching together concen- tric mosaics, omnidirectional video and light fields.

Full-text preview

Available from: psu.edu
  • Source
    [Show abstract] [Hide abstract] ABSTRACT: Interactive walkthrough applications require detailed 3D models to give users a sense of immersion in an environment. Traditionally these models are built using computer-aided design tools to define geometry and material properties. But creating detailed models is time-consuming and it is also difficult to reproduce all geometric and photometric subtleties of real-world scenes. Computer vision attempts to alleviate this problem by extracting geometry and photogrammetry from images of the real-world scenes. However, these models are still limited in the amount of detail they recover. Image-based rendering generates novel views by resampling a set of images of the environment without relying upon an explicit geometric model. Current such techniques limit the size and shape of the environment, and they do not lend themselves to walkthrough applications. In this paper, we define a parameterization of the 4D plenoptic function that is particularly suitable for interactive walkthroughs and define a method for its sampling and reconstructing. Our main contributions are: 1) a parameterization of the 4D plenoptic function that supports walkthrough applications in large, arbitrarily shaped environments; 2) a simple and fast capture process for complex environments; and 3) an automatic algorithm for reconstruction of the plenoptic function.
    Full-text · Conference Paper · Jan 2001
  • Source
    [Show abstract] [Hide abstract] ABSTRACT: Image-based rendering is a powerful new approach for generating real-time photorealistic computer graphics. It can provide convinc- ing animations without an explicit geometric representation. We use the "plenoptic function" of Adelson and Bergen to provide a concise problem statement for image-based rendering paradigms, such as morphing and view interpolation. The plenoptic function is a param- eterized function for describing everything that is visible from a given point in space. We present an image-based rendering system based on sampling, reconstructing, and resampling the plenoptic function. In addition, we introduce a novel visible surface algorithm and a geometric invariant for cylindrical projections that is equiva- lent to the epipolar constraint defined for planar projections.
    Preview · Conference Paper · Jan 1995
  • Source
    [Show abstract] [Hide abstract] ABSTRACT: In this paper, we present a novel image-based rendering technique, which we call manifold hopping. Our technique provides users with perceptually continuous navigation by using only a small number of strategically sampled manifold mosaics or multiperspective panoramas. Manifold hopping has two modes of navigation: moving continuously along any manifold, and discretely between manifolds. An important feature of manifold hopping is that significant data reduction can be achieved without sacrificing output visual fidelity, by carefully adjusting the hopping intervals. A novel view along the manifold is rendered by locally warping a single manifold mosaic using a constant depth assumption, without the need for accurate depth or feature correspondence. The rendering errors caused by manifold hopping can be analyzed in the signed Hough ray space. Experiments with real data demonstrate that we can navigate smoothly in a virtual environment with as little as 88k data compressed from 11 concentric mosaics.
    Full-text · Article · Oct 2002 · International Journal of Computer Vision
  • Source
    [Show abstract] [Hide abstract] ABSTRACT: This paper discusses a new method for capturing the complete appearanceof both synthetic and real world objects and scenes, representing this information, and then using this representation to render images of the object from new camera positions. Unlike the shape capture process traditionally used in computer vision and the rendering process traditionally used in computer graphics, our approach does not rely on geometric representations. Instead we sample and reconstruct a 4D function, which we call a Lumigraph. The Lumigraph is a subset of the complete plenoptic function that describes the flow of light at all positions in all directions. With the Lumigraph, newimages of the object canbe generated very quickly, independent of the geometric or illumination complexity of the scene or object. The paper discusses a complete working system including the capture of samples, the construction of the Lumigraph, and the subsequent rendering of images from this new representation.
    Full-text · Article · Aug 2001 · ACM SIGGRAPH Computer Graphics
  • [Show abstract] [Hide abstract] ABSTRACT: The problem we solve in this paper is the following. Suppose we are given N views of a static scene obtained from different viewpoints, perhaps with different cameras. These viewpoints we call reference viewpoints since they are all we know of the scene. We would like to decide if it is possible to predict ano- ther view of the scene taken by a camera from a viewpoint which is arbitrary and a priori di erent from all the reference viewpoints. One method for doing this would be to use these viewpoints to construct a three-dimensional repre- sentation of the scene and reproject this representation on the retinal plane of the virtual camera. In order to achieve this goal, we would have to establish some sort of calibration of our system of cameras, fuse the three-dimensional representations obtained from, say, pairs of cameras thereby obtaining a set of 3-D points, the scene. We would then have to approximate this set of points by surfaces, a segmentation problem which is still mostly unsolved, and then intersect the optical rays from the virtual camera with these sur- faces. This is the most straightforward way of going from a set of images to a new image using the current computer vision paradigm of rst building a three-dimensional representation of the environment from which the rest is derived. We do not claim that there does not exist any simpler way of using the three-dimensional representation than the one we just sketched, but this is just simply not our point. Our point is that it is possible to avoid entirely the explicit three-dimensional reconstruction process: the scene is represented by its images and by some ba- sically linear relations that govern the way points can be put in correspondence between views when they are the images of the same scene-point. These images and their algebraic relations are all we need for predicting a new image. This approach is similar in spirit to the one that has been used in trinocular stereo. Hypotheses of correspondences between two of the images are used to predict features in the third. These predictions can then be checked to validate or inva- lidate the initial correspondence. This approach has proved to be quite e cient and accurate. Related to these ideas are those develo- ped in the photogrammetric community under the name of transfer methods which nd for one or more image points in a given image set, the corresponding points in some new image set.
    No preview · Article · Jan 1994
  • [Show abstract] [Hide abstract] ABSTRACT: A traditional approach to extracting geometric information from a large scene is to compute multiple 3-D depth maps from stereo pairs or direct range finders, and then to merge the 3-D data This is not only computationally intensive, but the resulting merged depth maps may be subject to merging errors, especially if the relative poses between depth maps are not known exactly. The 3-D data may also have to be resampled before merging, which adds additional complexity and potential sources of errors. This paper provides a means of directly extracting 3-D data covering a very wide field of view, thus by-passing the need for numerous depth map merging. In our work, cylindrical images are first composited from sequences of images taken while the camera is rotated 360 ffi about a vertical axis. By taking such image panoramas at different camera locations, we can recover 3-D data of the scene using a set of simple techniques: feature tracking, an 8-point structure from motion algorithm, an...
    No preview · Article · Sep 1998
  • Source
    [Show abstract] [Hide abstract] ABSTRACT: experiment. Electrophysiologists have described neurons in striate cortex that are selectively sensitive to certain visual properties; for reviews, see Hubel (1988) and DeValois and DeValois (1988). Psychophysicists have inferred the existence of channels that are tuned for certain visual properties; for reviews, see Graham (1989), Olzak and Thomas (1986), Pokorny and Smith (1986), and Watson (1986). Researchers in perception have found aspects of visual stimuli that are processed pre-attentively (Beck, 1966; Bergen & Julesz, 1983; Julesz & Bergen, Motion Color Binocular disparity Retinal processing Early vision Memory Higher-level vision Etc... Retina More processing Still more processing Orientation Fig.1.1 A generic diagram for visual processing. In this approach, early vision consists of a set of parallel pathways, each analyzing some particular aspect of the visual stimulus. 1983; Treisman, 1986; Treisman & Gelade, 1980). And in computational
    Full-text · Article · Aug 1997
Show more