No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Non-rigid Registration with Reliable Distance Field for Dynamic Shape Completion
Abstract
We propose a non-rigid registration method for completion of dynamic shapes with occlusion. Our method is based on the idea that an occluded region in a certain frame should be visible in another frame and that local regions should be moving rigidly when the motion is small. We achieve this with a novel reliable distance field (DF) for non-rigid registration with missing regions. We first fit a pseudo-surface onto the input shape using a surface reconstruction method. We then calculate the difference between the DF of the input shape and the pseudo-surface. We define the areas with large difference as unreliable, as these areas indicate that the original shape cannot be found nearby. We then conduct non-rigid registration using local rigid transformations to match the source and target data at visible regions and maintain the original shape as much as possible in occluded regions. The experimental results demonstrate that our method is capable of accurately filling in the missing regions using the shape information from prior or posterior frames. By sequentially processing the data, our method is also capable of completing an entire sequence with missing regions.
ResearchGate has not been able to resolve any citations for this publication.
This paper proposes a non-rigid registration formulation capturing both global and local deformations in a single framework. This formulation is based on a quadratic estimation of the registration distance together with a quadratic regularization term. Hence, the optimal transformation parameters are easily obtained by solving a liner system of equations, which guarantee a fast convergence. Experimental results with challenging 2D and 3D shapes are presented to show the validity of the proposed framework. Furthermore, comparisons with the most relevant approaches are provided.
A novel correspondence-less approach is proposed to find a thin plate spline map between a pair of deformable 3D objects represented by triangular surface meshes. The proposed method works without landmark extraction and feature correspondences. The aligning transformation is found simply by solving a system of nonlinear equations. Each equation is generated by integrating a nonlinear function over the object's domains. We derive recursive formulas for the efficient computation of these integrals. Based on a series of comparative tests on a large synthetic dataset, our triangular mesh-based algorithm outperforms state of the art methods both in terms of computing time and accuracy. The applicability of the proposed approach has been demonstrated on the registration of 3D lung CT volumes.
In this paper, registration problem is formulated as a point to model distance minimization. Unlike most of the existing works, which are based on minimizing a point-wise correspondence term, this formulation avoids the correspondence search that is time-consuming. In the first stage the target set is described through an implicit function by employing a linear least squares fitting. This function can be either implicit polynomial (IP) or Implicit B-Spline (IBS) from a coarse to fine representation. In the second stage we show how the obtained implicit representation is used as an interface to convert point-to-point registration into point-to-implicit problem. Furthermore, we show that this registration distance is smooth and can be minimized by means of Levengberg-Marquardt algorithm. All the formulations presented for both stages are compact and easy to implement. Moreover, we show that our registration method can be handled by using any implicit representation though some are coarse and others provide finer representations; hence a tradeoff between speed and accuracy can be set by employing the right implicit function. Experimental results and comparisons in 2D and 3D show the robustness and the speed of convergence of the proposed approach.
In this paper, a novel method to solve the shape registration problem covering both global and local deformations is proposed. The vector distance function (VDF) is used to represent source and target shapes. The problem is formulated as an energy optimization process by matching the VDF's of the source and target shapes. The minimization process results in estimating the transformation parameters for the global and local deformation cases. Gradient descent optimization handles the computation of scaling, rotation, and translation matrices used to minimize the global differences between source and target shapes. Non rigid deformations require a large number of parameters which make the use of the gradient descent minimization a very time consuming process. We propose to compute the local deformation parameters using a closed form solution as a linear system of equations derived from approximating an objective function. Extensive experimental validations and comparisons performed on generalized 2D shape data demonstrate the robustness and effectiveness of the method.
In this paper, we present a unified framework for the rigid and nonrigid point set registration problem in the presence of significant amounts of noise and outliers. The key idea of this registration framework is to represent the input point sets using Gaussian mixture models. Then, the problem of point set registration is reformulated as the problem of aligning two Gaussian mixtures such that a statistical discrepancy measure between the two corresponding mixtures is minimized. We show that the popular iterative closest point (ICP) method and several existing point set registration methods in the field are closely related and can be reinterpreted meaningfully in our general framework. Our instantiation of this general framework is based on the the L2 distance between two Gaussian mixtures, which has the closed-form expression and in turn leads to a computationally efficient registration algorithm. The resulting registration algorithm exhibits inherent statistical robustness, has an intuitive interpretation, and is simple to implement. We also provide theoretical and experimental comparisons with other robust methods for point set registration.
We use a Hierarchical Partition of Unity Finite Element Method (H-PUFEM) to represent and analyse the non-rigid deformation fields involved in multidimensional image reg- istration. We make use of the Ritz-Galerkin direct varia- tional method to solve non-rigid image registration prob- lems with various deformation constraints. In this method, we directly seek a set of parameters that minimizes the objective function. We thereby avoid the loss of informa- tion that may occur when an Euler-Lagrange formulation is used. Experiments are conducted to demonstrate the ad- vantages of our approach when registering synthetic images having little of or no localizing features. As a special case, conformal mapping problems can be accurately solved in this manner. We also illustrate our approach with an appli- cation to Cardiac Magnetic Resonance temporal sequences.
Pose space deformation generalizes and improves upon both shape interpolation and common skeleton-driven deformation techniques. This deformation approach proceeds from the observation that several types of deformation can be uniformly represented as mappings from a pose space, defined by either an underlying skeleton or a more abstract system of parameters, to displacements in the object local coordinate frames. Once this uniform representation is identified, previously disparate deformation types can be accomplished within a single unified approach. The advantages of this algorithm include improved expressive power and direct manipulation of the desired shapes yet the performance associated with traditional shape interpolation is achievable. Appropriate applications include animation of facial and body deformation for entertainment, telepresence, computer gaming, and other applications where direct sculpting of deformations is desired or where real-time synthesis of a deforming model is required.
We present a robust and efficient algorithm for the pairwise non-rigid registration of partially overlapping 3D surfaces. Our approach treats non-rigid registration as an optimization problem and solves it by alternating between correspondence and deformation optimization. Assuming approximately isometric deformations, robust correspondences are generated using a pruning mechanism based on geodesic consistency. We iteratively learn an appropriate deformation discretization from the current set of correspondences and use it to update the correspondences in the next iteration. Our algorithm is able to register partially similar point clouds that undergo large deformations, in just a few seconds. We demonstrate the potential of our algorithm in various applications such as example based articulated segmentation, and shape interpolation.
Details in mesh animations are difficult to generate but they have great impact on visual quality. In this work, we demonstrate a practical software system for capturing such details from multi-view video recordings. Given a stream of synchronized video images that record a human performance from multiple viewpoints and an articulated template of the performer, our system captures the motion of both the skeleton and the shape. The output mesh animation is enhanced with the details observed in the image silhouettes. For example, a performance in casual loose-fitting clothes will generate mesh animations with flowing garment motions. We accomplish this with a fast pose tracking method followed by nonrigid deformation of the template to fit the silhouettes. The entire process takes less than sixteen seconds per frame and requires no markers or texture cues. Captured meshes are in full correspondence making them readily usable for editing operations including texturing, deformation transfer, and deformation model learning.
Of great importance to a wide variety of computer vision and image analysis problems is the ability to represent two- (2D) and three-dimensional (3D) data or objects. Implicit polynomial curves and surfaces are two of the most useful representations available. Their representational power is evidenced by their ability to smooth noisy data and to interpolate through sparse or missing data. Furthermore, their associated Euclidean and affine invariants are powerful discriminators, making implicit polynomials a computationally attractive technology for recognizing objects in arbitrary positions with respect to cameras or range sensors. In this paper, we introduce a completely new approach to fitting implicit polynomials to data. The algorithm represents a significant advancement of implicit polynomial technology for three important reasons. First, it is orders of magnitude faster than existing methods. Second, it has significantly better repeatability and numerical stability than current m...
In this paper, we address the problem of tracking the temporal evolution of arbitrary shapes observed in multi-camera setups. This is motivated by the ever growing number of applications that require consistent shape information along temporal sequences. The approach we propose considers a temporal sequence of independently reconstructed surfaces and iteratively deforms a reference mesh to fit these observations. To effectively cope with outlying and missing geometry, we introduce a novel probabilistic mesh deformation framework. Using generic local rigidity priors and accounting for the uncertainty in the data acquisition process, this framework effectively handles missing data, relatively large reconstruction artefacts and multiple objects. Extensive experiments demonstrate the effectiveness and robustness of the method on various 4D datasets.
In this paper, we consider the problem of tracking non-rigid surfaces and propose a generic data-driven mesh deformation framework. In contrast to methods using strong prior models, this framework assumes little on the observed surface and hence easily generalizes to most free-form surfaces while effectively handling large deformations. To this aim, the reference surface is divided into elementary surface cells or patches. This strategy ensures robustness by providing natural integration domains over the surface for noisy data, while enabling to express simple patch-level rigidity constraints. In addition, we associate to this scheme a robust numerical optimization that solves for physically plausible surface deformations given arbitrary constraints. In order to demonstrate the versatility of the proposed framework, we conducted experiments on open and closed surfaces, with possibly non-connected components, that undergo large deformations and fast motions. We also performed quantitative and qualitative evaluations in multi-cameras and monocular environments, and with different types of data including 2D correspondences and 3D point clouds.
Point set registration is a key component in many computer vision tasks. The goal of point set registration is to assign correspondences between two sets of points and to recover the transformation that maps one point set to the other. Multiple factors, including an unknown nonrigid spatial transformation, large dimensionality of point set, noise, and outliers, make the point set registration a challenging problem. We introduce a probabilistic method, called the Coherent Point Drift (CPD) algorithm, for both rigid and nonrigid point set registration. We consider the alignment of two point sets as a probability density estimation problem. We fit the Gaussian mixture model (GMM) centroids (representing the first point set) to the data (the second point set) by maximizing the likelihood. We force the GMM centroids to move coherently as a group to preserve the topological structure of the point sets. In the rigid case, we impose the coherence constraint by reparameterization of GMM centroid locations with rigid parameters and derive a closed form solution of the maximization step of the EM algorithm in arbitrary dimensions. In the nonrigid case, we impose the coherence constraint by regularizing the displacement field and using the variational calculus to derive the optimal transformation. We also introduce a fast algorithm that reduces the method computation complexity to linear. We test the CPD algorithm for both rigid and nonrigid transformations in the presence of noise, outliers, and missing points, where CPD shows accurate results and outperforms current state-of-the-art methods.
We present a framework and algorithms for robust geometry and motion reconstruction of complex deforming shapes. Our method makes use of a smooth template that provides a crude approximation of the scanned object and serves as a geometric and topological prior for reconstruction. Large-scale motion of the acquired object is recovered using a novel space-time adaptive, non-rigid registration method. Fine-scale details such as wrinkles and folds are synthesized with an efficient linear mesh deformation algorithm. Subsequent spatial and temporal filtering of detail coefficients allows transfer of persistent geometric detail to regions not observed by the scanner. We show how this two-scale process allows faithful recovery of small-scale shape and motion features leading to a high-quality reconstruction. We illustrate the robustness and generality of our algorithm on a variety of examples composed of different materials and exhibiting a large range of dynamic deformations. © 2009 ACM.
We present a novel, variational and statistical approach for shape registration. Shapes of interest are implicitly embedded in a higher-dimensional space of distance transforms. In this implicit embedding space, registration is formulated in a hierarchical manner: the Mutual Information criterion supports various transformation models and is optimized to perform global registration; then, a B-spline-based Incremental Free Form Deformations (IFFD) model is used to minimize a Sum-of-Squared-Differences (SSD) measure and further recover a dense local nonrigid registration field. The key advantage of such framework is twofold: (1) it naturally deals with shapes of arbitrary dimension (2D, 3D, or higher) and arbitrary topology (multiple parts, closed/open) and (2) it preserves shape topology during local deformation and produces local registration fields that are smooth, continuous, and establish one-to-one correspondences. Its invariance to initial conditions is evaluated through empirical validation, and various hard 2D/3D geometric shape registration examples are used to show its robustness to noise, severe occlusion, and missing parts. We demonstrate the power of the proposed framework using two applications: one for statistical modeling of anatomical structures, another for 3D face scan registration and expression tracking. We also compare the performance of our algorithm with that of several other well-known shape registration algorithms.
Three-dimensional detection and shape recovery of a nonrigid surface from video sequences require deformation models to effectively take advantage of potentially noisy image data. Here, we introduce an approach to creating such models for deformable 3D surfaces. We exploit the fact that the shape of an inextensible triangulated mesh can be parameterized in terms of a small subset of the angles between its facets. We use this set of angles to create a representative set of potential shapes, which we feed to a simple dimensionality reduction technique to produce low-dimensional 3D deformation models. We show that these models can be used to accurately model a wide range of deforming 3D surfaces from video sequences acquired under realistic conditions.
Implicit surface representation is well suited for surface reconstruction from a large amount of 3D noisy data points with non-uniform sampling density. Previous 3D non-rigid model registration methods can only be applied to the mesh or volume representations, but not directly to implicit surfaces. To our best knowledge, the previous 3D registration methods for implicit surfaces can only handle rigid transformation and they must keep the data points on the surface. In this paper, we propose a new 3D non-rigid registration algorithm to register two multi-level partition of unity (MPU) implicit surfaces with a variational formulation. The 3D non-rigid transformation between two implicit surfaces is a continuous deformation function, which is determined via an energy minimization procedure. Under the octree structure in the MPU surface, each leaf cell is transformed by an individual affine transformation associated with an energy that is related to the distance between two general quadrics. The proposed algorithm can directly register between two 3D implicit surfaces without sampling on the two signed distance functions or polygonalizing implicit surfaces, which makes our algorithm efficient in both computation and memory requirement. Experimental results on 3D human organ and sculpture models demonstrate the effectiveness of the proposed algorithm.
The author addresses the problem of parametric representation and estimation of complex planar curves in 2-D surfaces in 3-D, and nonplanar space curves in 3-D. Curves and surfaces can be defined either parametrically or implicitly, with the latter representation used here. A planar curve is the set of zeros of a smooth function of two variables x - y , a surface is the set of zeros of a smooth function of three variables x - y - z , and a space curve is the intersection of two surfaces, which are the set of zeros of two linearly independent smooth functions of three variables x - y - z For example, the surface of a complex object in 3-D can be represented as a subset of a single implicit surface, with similar results for planar and space curves. It is shown how this unified representation can be used for object recognition, object position estimation, and segmentation of objects into meaningful subobjects, that is, the detection of `interest regions' that are more complex than high curvature regions and, hence, more useful as features for object recognition
The decomposition of deformations by principal warps is demonstrated. The method is extended to deal with curving edges between landmarks. This formulation is related to other applications of splines current in computer vision. How they might aid in the extraction of features for analysis, comparison, and diagnosis of biological and medical images is indicated.
We study the problem of creating a complete model of a physical object. Although this may be possible using intensity images, we here use images which directly provide access to three dimensional information. The first problem that we need to solve is to find the transformation between the different views. Previous approaches either assume this transformation to be known (which is extremely difficult for a complete model), or compute it with feature matching (which is not accurate enough for integration). In this paper, we propose a new approach which works on range data directly, and registers successive views with enough overlapping area to get an accurate transformation between views. This is performed by minimizing a functional which does not require point-to-point matches. We give the details of the registration method and modelling procedure, and illustrate them on real range images of complex objects.
We present a new robust point matching algorithm (RPM) that can
jointly estimate the correspondence and non-rigid transformations
between two point-sets that may be of different sizes. The algorithm
utilizes the soft assign for the correspondence and the thin-plate
spline for the non-rigid mapping. Embedded within a deterministic
annealing framework, the algorithm can automatically reject a fraction
of the points as outliers. Experiments on both 2D synthetic point-sets
with varying degrees of deformation, noise and outliers, and on real 3D
sulcal point-sets (extracted from brain MRI) demonstrate the robustness
of the algorithm
This paper introduces a new method of registering point sets. The registration error is directly minimized using general-purpose non-linear optimization (the Levenberg–Marquardt algorithm). The surprising conclusion of the paper is that this technique is comparable in speed to the special-purpose Iterated Closest Point algorithm, which is most commonly used for this task. Because the routine directly minimizes an energy function, it is easy to extend it to incorporate robust estimation via a Huber kernel, yielding a basin of convergence that is many times wider than existing techniques. Finally, we introduce a data structure for the minimization based on the chamfer distance transform, which yields an algorithm that is both faster and more robust than previously described methods.
We present a new approach to the geometric alignment of a point cloud to a surface and to related registration problems. The standard algorithm is the familiar ICP algorithm. Here, we provide an alternative concept which relies on instantaneous kinematics and on the geometry of the squared distance function of a surface. The proposed algorithm exhibits faster convergence than ICP; this is supported both by results of a local convergence analysis and by experiments.
We propose a novel point signature based on the properties of the heat diffusion process on a shape. Our signature, called the Heat Kernel Signature (or HKS), is obtained by restricting the we ll-known heat kernel to the temporal domain. Remarkably we show that under certain mild assumptions, HKS ca ptures all of the information contained in the heat kernel, and characterizes the shape up to isometry. This mean s that the restriction to the temporal domain, on the one hand, makes HKS much more concise and easily comm ensurable, while on the other hand, it preserves all of the information about the intrinsic geometry of the shape. In addition, HKS inherits many useful properties from the heat kernel, which means, in particular, that it is stableunder perturbations of the shape. Our signature also provides a natural and efficiently computable multi-sca le way to capture information about neighborhoods of a given point, which can be extremely useful in many ap plications. To demonstrate the practical relevance of our signature, we present several methods for non-rig id multi-scale matching based on the HKS and use it to detect repeated structure within the same shape and across a colle ction of shapes.
We present the articulated global registration algorithm to reconstruct articulated 3D models from dynamic range scan sequences. This new algorithm aligns multiple range scans simultaneously to reconstruct a full 3D model from the geometry of these scans. Unlike other methods, we express the surface motion in terms of a reduced deformable model and solve for joints and skinning weights. This allows a user to interactively manipulate the reconstructed 3D model to create new animations.
We express the global registration as an optimization of both the alignment of the range scans and the articulated structure of the model. We employ a graph-based representation for the skinning weights that successfully handles difficult topological cases well. Joints between parts are estimated automatically and are used in the optimization to preserve the connectivity between parts. The algorithm also robustly handles difficult cases where parts suddenly disappear or reappear in the range scans. The global registration produces a more accurate registration compared to a sequential registration approach, because it estimates the articulated structure based on the motion observed in all input frames. We show that we can automatically reconstruct a variety of articulated models without the use of markers, user-placed correspondences, segmentation, or template model.
This paper describes a general purpose, representation independent
method for the accurate and computationally efficient registration of
3-D shapes including free-form curves and surfaces. The method handles
the full six-degrees of freedom and is based on the iterative closest
point (ICP) algorithm, which requires only a procedure to find the
closest point on a geometric entity to a given point. The ICP algorithm
always converges monotonically to the nearest local minimum of a
mean-square distance metric, and experience shows that the rate of
convergence is rapid during the first few iterations. Therefore, given
an adequate set of initial rotations and translations for a particular
class of objects with a certain level of 'shape complexity', one can
globally minimize the mean-square distance metric over all six degrees
of freedom by testing each initial registration. For examples, a given
'model' shape and a sensed 'data' shape that represents a major portion
of the model shape can be registered in minutes by testing one initial
translation and a relatively small set of rotations to allow for the
given level of model complexity. One important application of this
method is to register sensed data from unfixtured rigid objects with an
ideal geometric model prior to shape inspection. The described method is
also useful for deciding fundamental issues such as the congruence
(shape equivalence) of different geometric representations as well as
for estimating the motion between point sets where the correspondences
are not known. Experimental results show the capabilities of the
registration algorithm on point sets, curves, and surfaces.
This paper proposes a new marker-less approach to capturing human performances from multi-view video. Our algorithm can jointly reconstruct spatio-temporally coherent geometry, motion and textural surface appearance of actors that perform complex and rapid moves. Furthermore, since our algorithm is purely meshbased and makes as few as possible prior assumptions about the type of subject being tracked, it can even capture performances of people wearing wide apparel, such as a dancer wearing a skirt. To serve this purpose our method efficiently and effectively combines the power of surface- and volume-based shape deformation techniques with a new mesh-based analysis-through-synthesis framework. This framework extracts motion constraints from video and makes the laser-scan of the tracked subject mimic the recorded performance. Also small-scale time-varying shape detail is recovered by applying model-guided multi-view stereo to refine the model surface. Our method delivers captured performance data at higher level of detail, is highly versatile, and is applicable to many complex types of scenes that could not be handled by alternative marker-based or marker-free recording techniques.
Surface editing operations commonly require geometric details of the surface to be preserved as much as possible. We argue that geometric detail is an intrinsic property of a surface and that, consequently, surface editing is best performed by operating over an intrinsic surface representation. We provide such a representation of a surface, based on the Laplacian of the mesh, by encoding each vertex relative to its neighborhood. The Laplacian of the mesh is enhanced to be invariant to locally linearized rigid transformations and scaling. Based on this Laplacian representation, we develop useful editing operations: interactive free-form deformation in a region of interest based on the transformation of a handle, transfer and mixing of geometric details between two surfaces, and transplanting of a partial surface mesh onto another surface. The main computation involved in all operations is the solution of a sparse linear system, which can be done at interactive rates. We demonstrate the effectiveness of our approach in several examples, showing that the editing operations change the shape while respecting the structural geometric detail.
We present a registration algorithm for pairs of deforming and partial range scans that addresses the challenges of non-rigid registration within a single non-linear optimization. Our algorithm simultaneously solves for correspondences between points on source and target scans, confidence weights that measure the reliability of each correspondence and identify non-overlapping areas, and a warping field that brings the source scan into alignment with the target geometry. The optimization maximizes the region of overlap and the spatial coherence of the deformation while minimizing registration error. All optimization parameters are chosen automatically; hand-tuning is not necessary. Our method is not restricted to part-in-whole matching, but addresses the general problem of partial matching, and requires no explicit prior correspondences or feature points. We evaluate the performance and robustness of our method using scan data acquired by a structured light scanner and compare our method with existing non-rigid registration algorithms. © 2008 The Eurographics Association and Blackwell Publishing Ltd.
We propose a new adaptive space deformation method for interactive shape modeling. A novel energy formulation based on elastically coupled volumetric cells yields intuitive detail preservation even under large deformations. By enforcing rigidity of the cells, we obtain an extremely robust numerical solver for the resulting nonlinear optimization problem. Scalability is achieved using an adaptive spatial discretization that is decoupled from the resolution of the embedded object. Our approach is versatile and easy to implement, supports thin-shell and solid deformations of 2D and 3D objects, and is applicable to arbitrary sample-based representations, such as meshes, triangle soups, or point clouds. © 2007 The Eurographics Association and Blackwell Publishing Ltd.
We present a 3D shape-based object recognition system for
simultaneous recognition of multiple objects in scenes containing
clutter and occlusion. Recognition is based on matching surfaces by
matching points using the spin image representation. The spin image is a
data level shape descriptor that is used to match surfaces represented
as surface meshes. We present a compression scheme for spin images that
results in efficient multiple object recognition which we verify with
results showing the simultaneous recognition of multiple objects from a
library of 20 models. Furthermore, we demonstrate the robust performance
of recognition in the presence of clutter and occlusion through analysis
of recognition trials on 100 scenes