Article

3D Human Model Reconstruction from Sparse Uncalibrated Views

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

This paper presents a novel two-stage algorithm for reconstructing 3D human models wearing regular clothes from sparse uncalibrated views. The first stage reconstructs a coarse model with the help of a template model for human figures. A non-rigid dense correspondence algorithm is applied to generate denser correspondences than traditional feature descriptors. We fit the template model to the point cloud reconstructed from dense correspondences while enclosing it with the visual hull. In the second stage, the coarse model from the first stage is refined with geometric details, such as wrinkles, reconstructed from shading information. To successfully extract shading information for a surface with nonuniform reflectance, a hierarchical density based clustering algorithm is adapted to obtain high-quality pixel clusters. Geometric details reconstructed using our new shading extraction method exhibit superior quality. Our algorithm has been validated with images from an existing dataset as well as images captured by a cell phone camera.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Another way to build 3D models is to use laser scanners [2], which are tools that can analyze objects or scenes in order to collect data about the shapes and appearances. After that the collected data would be used to build the 3D model [3]. ...
... Since that, photography could make it easier in case of capturing the view from multi view of points. Until now, there has been a desire for 3D model creation, but still many current 3D model reconstruction tools and methods lack high accuracy [3]. Usually, passive methods use two synchronized cameras. ...
... Subsequently, various 3D mannequins are obtained using the method of surface reconstruction based on feature curves [12]. Some other researchers are based on the silhouettes and body shapes on different views using the captured human body images [9]. Boisvert et al. [1] reconstruct the shape of a subject combining geometric with statistical priors from a frontal and lateral silhouettes of target human body. ...
Article
Full-text available
Personalized human shape plays an important role in system of virtual garment customization. In this paper, a new entity based multi-direction cooperative deformation algorithm (MDCD) is proposed to generate personalized human shape. We primarily extract the description parameters of entities using different direction sections based on unified division method. By introducing the entity curve interpolation and correspondence between elementary and target entities, the generated distortion of various human entities from simulated deformation function will be avoided. With the good capacity of accuracy and reusability of human entities, the MDCD algorithm can greatly improve the generation efficiency of personalized human shapes. The elementary entities on template model are firstly located using the division method of Poser, and description parameters on entities are then extracted based on the entity direction. Thus, shape characteristics can be represented by a set of direction curves on different angles by entity curve interpolation. Subsequently shape characteristics on personalized human body are generated by a comparison of direction curves of template and target entities. Finally, the MDCD algorithm is applied for generating personalized entities and shapes. Experiments show that compared with the existed algorithms, personalized human shapes generated by our method are smoother and more similar to the template model.
Article
Full-text available
This work introduces a novel method for the creation of an in-home virtual dressing room for garment fitting using an integrated system consisting of personalized 3D body model reconstruction and garment fitting simulation. Our method gives saliency to and establishes a relational interconnection between the massive scatter points on the 3D generic model. Starting with a small set of anthropometric interconnected ordered intrinsic control points residing on the silhouette of the projections of the generic model, corresponding control points on two canonical images of the person are automatically found – hence importing equivalent saliency between the two sets. Further equivalent saliencies between the projected points from the generic model and the canonical images are established through a loop subdivision process. Human shape mesh personalization is done through morphing the points on the generic model to follow and be consistent with their equivalent points on the canonical images. The 3D reconstruction yields sub resolution errors (high level accuracy) when compared to the average resolution of the original model using the CAESAR dataset. The reconstructed model is then fitted with garments sized to the 3D personalized model given at least one frontal image of the garment with no requirement for a full 3D view of the garment. Our method can also be applied to virtual fitting system for online stores and/or for clothing design and personalized garment simulation. The method is convenient, simple, efficient, and requires no intervention from the user aside from taking two images with a camera or smart phone.
Article
Full-text available
Points acquired by laser scanners are not intrinsically equipped with nor-mals, which are essential to surface reconstruction and point set render-ing using surfels. Normal estimation is notoriously sensitive to noise. Near sharp features, the computation of noise-free normals becomes even more challenging due to the inherent under-sampling problem at edge singular-ities. As a result, common edge-aware consolidation techniques such as bilateral smoothing may still produce erroneous normals near the edges. We propose a resampling approach to process a noisy and possibly outlier-ridden point set in an edge-aware manner. Our key idea is to first resample away from the edges so that reliable normals can be computed at the sam-ples, and then based on reliable data, we progressively resample the point set while approaching the edge singularities. We demonstrate that our edge-aware resampling (EAR) algorithm is capable of producing consolidated point sets with noise-free normals and clean preservation of sharp features. Authors' addresses: copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permis-sion and/or a fee. Permissions may be requested from Publications Dept. We also show that EAR leads to improved performance of edge-aware re-construction methods and point set rendering techniques.
Article
Full-text available
3D scanning pose change output reconstruction textured reconstruction large variety of examples 3D print Figure 1: With our system, users can scan themselves with a single 3D sensor by rotating the same pose for a few different views (typically eight, ⇠45 degrees apart) to cover the full body. Our method robustly registers and merges different scans into a watertight surface with consistent texture in spite of shape changes during repositioning, and lighting differences between the scans. These surfaces are suitable for applications such as online avatars or 3D printing (the miniature shown here was printed using a ZPrinter 650.) Abstract We develop an automatic pipeline that allows ordinary users to cap-ture complete and fully textured 3D models of themselves in min-utes, using only a single Kinect sensor, in the uncontrolled lighting environment of their own home. Our method requires neither a turntable nor a second operator, and is robust to the small defor-mations and changes of pose that inevitably arise during scanning. After the users rotate themselves with the same pose for a few scans from different views, our system stitches together the cap-tured scans using multi-view non-rigid registration, and produces watertight final models. To ensure consistent texturing, we re-cover the underlying albedo from each scanned texture and generate seamless global textures using Poisson blending. Despite the mini-mal requirements we place on the hardware and users, our method is suitable for full body capture of challenging scenes that cannot be handled well using previous methods, such as those involving loose clothing, complex poses, and props.
Article
Full-text available
This paper presents a robust multiview stereo (MVS) algorithm for free-viewpoint video. Our MVS scheme is totally point-cloud-based and consists of three stages: point cloud extraction, merging, and meshing. To guarantee reconstruction accuracy, point clouds are first extracted according to a stereo matching metric which is robust to noise, occlusion, and lack of texture. Visual hull information, frontier points, and implicit points are then detected and fused with point fidelity information in the merging and meshing steps. All aspects of our method are designed to counteract potential challenges in MVS data sets for accurate and complete model reconstruction. Experimental results demonstrate that our technique produces the most competitive performance among current algorithms under sparse viewpoint setups according to both static and motion MVS data sets.
Article
Full-text available
Depth camera such as Microsoft Kinect, is much cheaper than conventional 3D scanning devices, and thus it can be acquired for everyday users easily. However, the depth data captured by Kinect over a certain distance is of extreme low quality. In this paper, we present a novel scanning system for capturing 3D full human body models by using multiple Kinects. To avoid the interference phenomena, we use two Kinects to capture the upper part and lower part of a human body respectively without overlapping region. A third Kinect is used to capture the middle part of the human body from the opposite direction. We propose a practical approach for registering the various body parts of different views under non-rigid deformation. First, a rough mesh template is constructed and used to deform successive frames pairwisely. Second, global alignment is performed to distribute errors in the deformation space, which can solve the loop closure problem efficiently. Misalignment caused by complex occlusion can also be handled reasonably by our global alignment algorithm. The experimental results have shown the efficiency and applicability of our system. Our system obtains impressive results in a few minutes with low price devices, thus is practically useful for generating personalized avatars for everyday users. Our system has been used for 3D human animation and virtual try on, and can further facilitate a range of home–oriented virtual reality (VR) applications.
Conference Paper
Full-text available
This paper proposes a novel algorithm for calibrated multi-view stereopsis that outputs a (quasi) dense set of rectan- gular patches covering the surfaces visible in the input images. This algorithm does not require any initialization in the form of a bounding volume, and it detects and discards automatically out- liers and obstacles. It does not perform any smoothing across nearby features, yet is currently the top performer in terms of both coverage and accuracy for four of the six benchmarkdatasets pre- sented in (20). The keys to its performance are effective tech- niques for enforcing local photometric consistency and global visibility constraints. Stereopsis is implemented as a match, ex- pand, and filter procedure, starting from a sparse set of matched keypoints, and repeatedly expanding these to nearby pixel corre- spondences before using visibility constraints to filter away false matches. A simple but effective method for turning the resulting patch model into a mesh appropriate for image-based modeling is also presented. The proposed approach is demonstrated on vari- ous datasets including objects with fine surface details, deep con- cavities, and thin structures, outdoor scenes observed from a re- stricted set of viewpoints, and "crowded" scenes where moving obstacles appear in different places in multiple images of a static structure of interest.
Conference Paper
Full-text available
We present an approach to add true fine-scale spatio-temporal shape detail to dynamic scene geometry captured from multi-view video footage. Our approach exploits shading information to recover the millimeter-scale surface structure, but in contrast to related approaches succeeds under general unconstrained lighting conditions. Our method starts off from a set of multi-view video frames and an initial series of reconstructed coarse 3D meshes that lack any surface detail. In a spatio-temporal maximum a posteriori probability (MAP) inference framework, our approach first estimates the incident illumination and the spatially-varying albedo map on the mesh surface for every time instant. Thereafter, albedo and illumination are used to estimate the true geometric detail visible in the images and add it to the coarse reconstructions. The MAP framework uses weak temporal priors on lighting, albedo and geometry which improve reconstruction quality yet allow for temporal variations in the data.
Conference Paper
Full-text available
We describe a solution to the challenging problem of es- timating human body shape from a single photograph or painting. Our approach computes shape and pose parame- ters of a 3D human body model directly from monocular im- age cues and advances the state of the art in several direc- tions. First, given a user-supplied estimate of the subject's height and a few clicked points on the body we estimate an initial 3D articulated body pose and shape. Second, using this initial guess we generate a tri-map of regions inside, outside and on the boundary of the human, which is used to segment the image using graph cuts. Third, we learn a low-dimensional linear model of human shape in which variations due to height are concentrated along a single dimension, enabling height-constrained estimation of body shape. Fourth, we formulate the problem of parametric hu- man shape from shading. We estimate the body pose, shape and reflectance as well as the scene lighting that produces a synthesized body that robustly matches the image evidence. Quantitative experiments demonstrate how smooth shading provides powerful constraints on human shape. We further demonstrate a novel application in which we extract 3D hu- man models from archival photographs and paintings.
Conference Paper
Full-text available
Most of the algorithms dealing with image based 3-D reconstruc- tion involve the evolution of a surface based on a minimization criterion. The mesh parametrization, while allowing for an accurate surface representation, suf- fers from the inherent problems of not being able to reliably deal with self- intersections and topology changes. As a consequence, an important number of methods choose implicit representations of surfaces, e.g. level set methods, that naturally handle topology changes and intersections. Nevertheless, these methods rely on space discretizations, which introduce an unwanted precision-complexity trade-off. In this paper we explore a new mesh-based solution that robustly han- dles topology changes and removes self intersections, therefore overcoming the traditional limitations of this type of approaches. To demonstrate its efficiency, we present results on 3-D surface reconstruction from multiple images and compare them with state-of-the art results.
Article
Full-text available
We present an easy-to-use image retouching technique for realistic reshaping of human bodies in a single image. A model-based approach is taken by integrating a 3D whole-body morphable model into the reshaping process to achieve globally consistent editing effects. A novel body-aware image warping approach is introduced to reliably transfer the reshaping effects from the model to the image, even under moderate fitting errors. Thanks to the parametric nature of the model, our technique parameterizes the degree of reshaping by a small set of semantic attributes, such as weight and height. It allows easy creation of desired reshaping effects by changing the full-body attributes, while producing visually pleasing results even for loosely-dressed humans in casual photographs with a variety of poses and shapes.
Article
Full-text available
This paper presents a new efficient method for recovering reliable local sets of dense correspondences between two images with some shared content. Our method is designed for pairs of images depicting similar regions acquired by different cameras and lenses, under non-rigid transformations, under different lighting, and over different backgrounds. We utilize a new coarse-to-fine scheme in which nearest-neighbor field computations using Generalized PatchMatch [Barnes et al. 2010] are interleaved with fitting a global non-linear parametric color model and aggregating consistent matching regions using locally adaptive constraints. Compared to previous correspondence approaches, our method combines the best of two worlds: It is dense, like optical flow and stereo reconstruction methods, and it is also robust to geometric and photometric variations, like sparse feature matching. We demonstrate the usefulness of our method using three applications for automatic example-based photograph enhancement: adjusting the tonal characteristics of a source image to match a reference, transferring a known mask to a new image, and kernel estimation for image deblurring.
Article
Full-text available
This paper proposes a novel algorithm for multiview stereopsis that outputs a dense set of small rectangular patches covering the surfaces visible in the images. Stereopsis is implemented as a match, expand, and filter procedure, starting from a sparse set of matched keypoints, and repeatedly expanding these before using visibility constraints to filter away false matches. The keys to the performance of the proposed algorithm are effective techniques for enforcing local photometric consistency and global visibility constraints. Simple but effective methods are also proposed to turn the resulting patch model into a mesh which can be further refined by an algorithm that enforces both photometric consistency and regularization constraints. The proposed approach automatically detects and discards outliers and obstacles and does not require any initialization in the form of a visual hull, a bounding box, or valid depth ranges. We have tested our algorithm on various data sets including objects with fine surface details, deep concavities, and thin structures, outdoor scenes observed from a restricted set of viewpoints, and "crowded" scenes where moving obstacles appear in front of a static structure of interest. A quantitative evaluation on the Middlebury benchmark shows that the proposed method outperforms all others submitted so far for four out of the six data sets.
Article
We present a system for interactively browsing and exploring large unstructured collections of photographs of a scene using a novel 3D interface. Our system consists of an image-based modeling front end that automatically computes the viewpoint of each photograph as well as a sparse 3D model of the scene and image to model correspondences. Our photo explorer uses image-based rendering techniques to smoothly transition between photographs, while also enabling full 3D navigation and exploration of the set of images and world geometry, along with auxiliary information such as overhead maps. Our system also makes it easy to construct photo tours of scenic or historic locations, and to annotate image details, which are automatically transferred to other relevant images. We demonstrate our system on several large personal photo collections as well as images gathered from Internet photo sharing sites.
Conference Paper
We propose a theoretically and practically improved density-based, hierarchical clustering method, providing a clustering hierarchy from which a simplified tree of significant clusters can be constructed. For obtaining a “flat” partition consisting of only the most significant clusters (possibly corresponding to different density thresholds), we propose a novel cluster stability measure, formalize the problem of maximizing the overall stability of selected clusters, and formulate an algorithm that computes an optimal solution to this problem. We demonstrate that our approach outperforms the current, state-of-the-art, density-based clustering methods on a wide variety of real world data.
Conference Paper
Next to lighting, posing is the most challenging aspect of portrait photography. A commonly adopted solution is to learn by example, which is beneficial for both trained photographers and novice users, especially when subjects have no clue about how to pose themselves. A collection of portrait images by professionals (e.g., [Perkins 2009]) provides a resource for photographers seeking inspiration for their own work. Such handful posing references (e.g., Posing App) have also been made available to smartphone platforms, which offer the unique possibility of directly overlaying camera view with a reference pose as visual guidance.
Conference Paper
We present a novel framework to estimate detailed shape of diffuse objects with uniform albedo from a single RGB-D image. To estimate accurate lighting in natural illumination environment, we introduce a general lighting model consisting of two components: global and local models. The global lighting model is estimated from the RGB-D input using the low-dimensional characteristic of a diffuse reflectance model. The local lighting model represents spatially varying illumination and it is estimated by using the smoothly-varying characteristic of illumination. With both the global and local lighting model, we can estimate complex lighting variations in uncontrolled natural illumination conditions accurately. For high quality shape capture, a shape-from-shading approach is applied with the estimated lighting model. Since the entire process is done with a single RGB-D input, our method is capable of capturing the high quality shape details of a dynamic object under natural illumination. Experimental results demonstrate the feasibility and effectiveness of our method that dramatically improves shape details of the rough depth input.
We present a shading-based shape refinement algorithm which uses a noisy, incomplete depth map from Kinect to help resolve ambiguities in shape-from-shading. In our framework, the partial depth information is used to overcome bas-relief ambiguity in normals estimation, as well as to assist in recovering relative albedos, which are needed to reliably estimate the lighting environment and to separate shading from albedo. This refinement of surface normals using a noisy depth map leads to high-quality 3D surfaces. The effectiveness of our algorithm is demonstrated through several challenging real-world examples.
Article
We present a new performance capture approach that incorporates a physically-based cloth model to reconstruct a rigged fullyanimatable virtual double of a real person in loose apparel from multi-view video recordings. Our algorithm only requires a minimum of manual interaction. Without the use of optical markers in the scene, our algorithm first reconstructs skeleton motion and detailed time-varying surface geometry of a real person from a reference video sequence. These captured reference performance data are then analyzed to automatically identify non-rigidly deforming pieces of apparel on the animated geometry. For each piece of apparel, parameters of a physically-based real-time cloth simulation model are estimated, and surface geometry of occluded body regions is approximated. The reconstructed character model comprises a skeleton-based representation for the actual body parts and a physically-based simulation model for the apparel. In contrast to previous performance capture methods, we can now also create new real-time animations of actors captured in general apparel.
Multi-view stereo methods reconstruct 3D geometry from images well for sufficiently textured scenes, but often fail to recover high-frequency surface detail, particularly for smoothly shaded surfaces. On the other hand, shape-from-shading methods can recover fine detail from shading variations. Unfortunately, it is non-trivial to apply shape-from-shading alone to multi-view data, and most shading-based estimation methods only succeed under very restricted or controlled illumination. We present a new algorithm that combines multi-view stereo and shading-based refinement for high-quality reconstruction of 3D geometry models from images taken under constant but otherwise arbitrary illumination. We have tested our algorithm on several scenes that were captured under several general and unknown lighting conditions, and we show that our final reconstructions rival laser range scans.
We propose tensor-based multiview stereo (TMVS) for quasi-dense D reconstruction from uncalibrated images. Our work is inspired by the patch-based multiview stereo (PMVS), a state-of-the-art technique in multiview stereo reconstruction. The effectiveness of PMVS is attributed to the use of 3D patches in the match-propagate-filter MVS pipeline. Our key observation is: PMVS has not fully utilized the valuable 3D geometric cue available in 3D patches which are oriented points. This paper combines the complementary advantages of photoconsistency, visibility and geometric consistency enforcement in MVS via the use of 3D tensors, where our closed-form solution to tensor voting provides a unified approach to implement the match-propagate-filter pipeline. Using PMVS as the implementation backbone where TMVS is built, we provide qualitative and quantitative evaluation to demonstrate how TMVS significantly improve the MVS pipeline.
Conference Paper
Occlusion contours are a natural feature to draw when tracing an object in an image or when drawing an object. We investigate the development of 3D models from multi-stroke contour drawings with the help of a 3D template model that serves as a shape prior. The template is aligned and then deformed by our method to match the drawn contours. At the heart of this process is the need to provide good correspondences between points on the contours and vertices on the model, which we pose as an optimisation problem using a hidden Markov model. An alternating correspond-and-deform process then progressively deforms the 3D template to match the image contours. We demonstrate the method on a wide range of examples.
Article
We present a deformable model to reconstruct a surface from a point cloud. The model is based on an explicit mesh representation composed of multiple competing evolving fronts. These fronts adapt to the local feature size of the target shape in a coarse–to–fine manner. Hence, they approach towards the finer (local) features of the target shape only after the reconstruction of the coarse (global) features has been completed. This conservative approach leads to a better control and interpretation of the reconstructed topology. The use of an explicit representation for the deformable model guarantees water-tightness and simple tracking of topological events. Furthermore, the coarse–to–fine nature of reconstruction enables adaptive handling of non-homogenous sample density, including robustness to missing data in defected areas. Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Digitizing and scanning. Keywords: surface reconstruction, deformable models
Article
is that these approaches will one day allow virtual tourism of the world's interesting and important sites. We present a system for interactively browsing and exploring large unstructured collections of photographs of a scene using a novel 3D interface. Our system consists of an image-based modeling front end that automatically computes the viewpoint of each photo-graph as well as a sparse D model of the scene and image to model correspondences. Our photo explorer uses image-based rendering techniques to smoothly transition between photographs, while also enabling full D navigation and exploration of the set of images and world geometry, along with auxiliary information such as overhead maps. Our system also makes it easy to construct photo tours of scenic or historic locations, and to annotate image details, which are automatically transferred to other relevant images. We demon-strate our system on several large personal photo collections as well as images gathered from Internet photo sharing sites. CR Categories: H. 5.1 [Information Interfaces and Presentation]: Multimedia Information Systems, Arti cial, augmented, and vir-tual realities I. 2.10 [Arti cial Intelligence]: Vision and Scene Understanding, Modeling and recovery of physical attributes Keywords: image-based rendering, image-based modeling, photo browsing, structure from motion
Article
We present an interactive system that lets a user move and deform a two-dimensional shape without manually establishing a skeleton or freeform deformation (FFD) domain beforehand. The shape is represented by a triangle mesh and the user moves several vertices of the mesh as constrained handles. The system then computes the positions of the remaining free vertices by minimizing the distortion of each triangle. While physically based simulation or iterative refinement can also be used for this purpose, they tend to be slow. We present a two-step closed-form algorithm that achieves real-time interaction. The first step finds an appropriate rotation for each triangle and the second step adjusts its scale. The key idea is to use quadratic error metrics so that each minimization problem becomes a system of linear equations. After solving the simultaneous equations at the beginning of interaction, we can quickly find the positions of free vertices during interactive manipulation. Our approach successfully conveys a sense of rigidity of the shape, which is difficult in space-warp approaches. With a multiple-point input device, even beginners can easily move, rotate, and deform shapes at will.
Article
Animating an articulated 3D character currently requires manual rigging to specify its internal skeletal structure and to define how the input motion deforms its surface. We present a method for animating characters automatically. Given a static character mesh and a generic skeleton, our method adapts the skeleton to the character and attaches it to the surface, allowing skeletal motion data to animate the character. Because a single skeleton can be used with a wide range of characters, our method, in conjunction with a library of motions for a few skeletons, enables a user-friendly animation system for novices and children. Our prototype implementation, called Pinocchio, typically takes under a minute to rig a character on a modern midrange PC.
This paper proposes a method for capturing the performance of a human or an animal from a multi-view video sequence. Given an articulated template model and silhouettes from a multi-view image sequence, our approach recovers not only the movement of the skeleton, but also the possibly non-rigid temporal deformation of the 3D surface. While large scale deformations or fast movements are captured by the skeleton pose and approximate surface skinning, true small scale deformations or non-rigid garment motion are captured by fitting the surface to the silhouette. We further propose a novel optimization scheme for skeleton-based pose estimation that exploits the skeleton's tree structure to split the optimization problem into a local one and a lower dimensional global one. We show on various sequences that our approach can capture the 3D motion of animals and humans accurately even in the case of rapid movements and wide apparel like skirts.
Article
This paper proposes a new marker-less approach to capturing human performances from multi-view video. Our algorithm can jointly reconstruct spatio-temporally coherent geometry, motion and textural surface appearance of actors that perform complex and rapid moves. Furthermore, since our algorithm is purely meshbased and makes as few as possible prior assumptions about the type of subject being tracked, it can even capture performances of people wearing wide apparel, such as a dancer wearing a skirt. To serve this purpose our method efficiently and effectively combines the power of surface- and volume-based shape deformation techniques with a new mesh-based analysis-through-synthesis framework. This framework extracts motion constraints from video and makes the laser-scan of the tracked subject mimic the recorded performance. Also small-scale time-varying shape detail is recovered by applying model-guided multi-view stereo to refine the model surface. Our method delivers captured performance data at higher level of detail, is highly versatile, and is applicable to many complex types of scenes that could not be handled by alternative marker-based or marker-free recording techniques.
Article
This paper proposes a quasi-dense approach to 3D surface model acquisition from uncalibrated images. First, correspondence information and geometry are computed based on new quasi-dense point features that are resampled subpixel points from a disparity map. The quasi-dense approach gives more robust and accurate geometry estimations than the standard sparse approach. The robustness is measured as the success rate of full automatic geometry estimation with all involved parameters fixed. The accuracy is measured by a fast gauge-free uncertainty estimation algorithm. The quasi-dense approach also works for more largely separated images than the sparse approach, therefore, it requires fewer images for modeling. More importantly, the quasidense approach delivers a high density of reconstructed 3D points on which a surface representation can be reconstructed. This fills the gap of insufficiency of the sparse approach for surface reconstruction, essential for modeling and visualization applications. Second, surface reconstruction methods from the given quasi-dense geometry are also developed. The algorithm optimizes new unified functionals integrating both 3D quasi-dense points and 2D image information, including silhouettes. Combining both 3D data and 2D images is more robust than the existing methods using only 2D information or only 3D data. An efficient bounded regularization method is proposed to implement the surface evolution by level-set methods. Its properties are discussed and proven for some cases. As a whole, a complete automatic and practical system of 3D modeling from raw images captured by hand-held cameras to surface representation is proposed. Extensive experiments demonstrate the superior performance of the quasi-dense approach with respect to the standard sparse approach in robustness, accuracy, and applicability.
  • H Li
  • E Vouga
  • A Gudym
  • L Luo
  • J T Barron
  • G Gusev
H. Li, E. Vouga, A. Gudym, L. Luo, J. T. Barron, and G. Gusev, " 3d self-portraits, " ACM Transactions on Graphics (Proceedings SIGGRAPH Asia 2013), vol. 32, no. 6, 2013.
A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video
  • Q Liu
  • W Dai
  • Xu
Feature Article 3. Y. Liu, Q. Dai, and W. Xu, "A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video," IEEE Trans. Visualization and Computer Graphics, vol. 16, no. 3, 2010, pp. 407-418.