-
[show abstract]
[hide abstract]
ABSTRACT: In this paper, we address the problem of segmenting data defined on a manifold into a set of regions with uniform properties. In particular, we propose a numerical method when the manifold is represented by a triangular mesh. Based on recent image segmentation models, our method minimizes a convex energy and then enjoys significant favorable properties: it is robust to initialization and avoid the problem of the existence of local minima present in many variational models. The contributions of this paper are threefold: firstly we adapt the convex image labelling model to manifolds; in particular the total variation formulation. Secondly we show how to implement the proposed method on triangular meshes, and finally we show how to use and combine the method in other computer vision problems, such as 3D reconstruction. We demonstrate the efficiency of our method by testing it on various data.
-
[show abstract]
[hide abstract]
ABSTRACT: In this paper, we address the problem of segmenting data defined on a manifold into a set of regions with uniform properties. In particular, we propose a numerical method when the manifold is represented by a triangular mesh. Based on recent image segmentation models, our method minimizes a convex energy and then enjoys significant favorable properties: it is robust to initialization and avoid the problem of the existence of local minima present in many variational models. The contributions of this paper are threefold: firstly we adapt the convex image labeling model to manifolds; in particular the total variation formulation. Secondly we show how to implement the proposed method on triangular meshes, and finally we show how to use and combine the method in other computer vision problems, such as 3D reconstruction. We demonstrate the efficiency of our method by testing it on various data.
17ème Congrès de Reconnaissance des Formes et Intelligence Artificielle.
-
[show abstract]
[hide abstract]
ABSTRACT: This paper gives an overview of works done in our group on 3D and appearance modeling of objects, from images. The backbone of our approach is to use what we consider as the principled optimization criterion for this problem: to maximize photoconsistency between input images and images rendered from the estimated surface geometry and appearance. In initial works, we have derived a general solution for this, showing how to write the gradient for this cost function (a non-trivial undertaking). In subsequent works, we have applied this solution to various scenarios: recovery of textured or uniform Lambertian or non-Lambertian surfaces, under static or varying illumination and with static or varying viewpoint. Our approach can be applied to these different cases, which is possible since it naturally merges cues that are often considered separately: stereo information, shading, silhouettes. This merge naturally happens as a result of the cost function used: when rendering estimated geometry and appearance (given known lighting conditions), the resulting images automatically contain these cues and their comparison with the input images thus implicitly uses these cues simultaneously.
Proceedings of the 14th Iberoamerican Conference on Pattern Recognition: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications.
-
[show abstract]
[hide abstract]
ABSTRACT: Helmholtz stereovision methods are limited to binocular stereovision or depth maps reconstruction. In this paper, we extend these methods to recover the full 3D shape of the objects of a scene from multiview Helmholtz stereopsis. Thus, we are able to reconstruct the complete three-dimensional shape of objects made of any arbitrary and unknown bidirectional reflectance distribution function. Unlike previous methods, this can be achieved using a full surface representation model. In particular occlusions (self occlusions as well as cast shadows) are easier to handle in the surface optimization process. More precisely, we use a triangular mesh representation which allows to naturally specify relationships between the geometry of a point of the scene and its surface normal. We show how to implement the presented approach using a coherent gradient descent flow. Results and benefits are illustrated on various examples.
Asian Conference on Computer Vision.
-
[show abstract]
[hide abstract]
ABSTRACT: In this paper we present a novel method to apply photometric stereo on textured dynamic surfaces. We aim at exploiting the high accuracy of photometric stereo and reconstruct local surface orientation from illumination changes. The main difficulty derives from the fact that photometric stereo requires varying illumination while the object remains still, which makes it quite impractical to use for dynamic surfaces. Using coloured lights gives a clear solution to this problem; however, the system of equations is still ill-posed and it is ambiguous whether the change of an observed surface colour is due to the change of the surface gradient or of the surface reflectance. In order to separate surface orientation from reflectance, our method tracks texture changes over time and exploits surface reflectance's temporal constancy. This additional constraint allows us to reformulate the problem as an energy functional minimisation, solved by a standard quasi-Newton method. Our method is tested both on real and synthetic data, quantitatively evaluated and compared to a state-of-the-art method.
Asian Conference on Computer Vision.
-
[show abstract]
[hide abstract]
ABSTRACT: This article proposes a variational multi-view stereo vision method based on meshes for recovering 3D scenes (shape and radiance) from images. Our method is based on generative models and minimizes the reprojection error (difference between the observed images and the images synthesized from the reconstruction). Our contributions are twofold. 1) For the first time, we rigorously compute the gradient of the reprojection error for non smooth surfaces defined by discrete triangular meshes. The gradient correctly takes into account the visibility changes that occur when a surface moves; this forces the contours generated by the reconstructed surface to perfectly match with the apparent contours in the input images. 2) We propose an original modification of the Lambertian model to take into account deviations from the constant brightness assumption without explicitly modelling the reflectance properties of the scene or other photometric phenomena involved by the camera model. Our method is thus able to recover the shape and the diffuse radiance of non Lambertian scenes.
-
[show abstract]
[hide abstract]
ABSTRACT: Cet article propose une méthode variationnelle basée sur les maillages pour la reconstruction 3D de scènes (forme et radiance) à partir de plusieurs images. Notre méthode est basée sur les modèles génératifs et minimise l'erreur de reprojection (différence entre une image observée et une image obtenue à partir de la reconstruction) par une descente de gradient. Pour la première fois, nous calculons le gradient de l'erreur de reprojection pour des surfaces non lisses représentées de manière discrète par des maillages triangulés. Le gradient prend correctement en compte les changements de visibilité qui apparaissent lorsque la surface bouge durant l'évolution; cela force les contours occultants générés par la surface à correspondre parfaitement aux contours apparents dans les images observées. Notre méthode est capable de retrouver la forme et la radiance de diverses scènes. This article proposes a variational multi-view stereo vision method based on meshes for recovering 3D scenes (shape and radiance) from images. Our method is based on generative models and minimizes the reprojection error (difference between the observed images and the images synthesized from the reconstruction). For the first time, we rigorously compute the gradient of the reprojection error for non smooth surfaces defined by discrete triangular meshes. The gradient correctly takes into account the visibility changes that occur when a surface moves; this forces the contours generated by the reconstructed surface to perfectly match with the apparent contours in the input images. Our method is thus able to recover the shape and the diffuse radiance of various scenes.
-
[show abstract]
[hide abstract]
ABSTRACT: We develop a variational method to recover both the shape and the reflectance of a scene surface(s) using multiple images, assuming that illumination conditions are fixed and known in advance. Scene and image formation are modeled with known information about cameras and illuminants, and scene recovery is achieved by minimizing a global cost functional with respect to both shape and reflectance. Unlike most previous methods recovering only the shape of Lambertian surfaces, the proposed method considers general dichromatic surfaces. We verify the method using synthetic data sets containing specular reflection.
-
[show abstract]
[hide abstract]
ABSTRACT: In this paper, we present a variational method that recovers both the shape and the reflectance of the Lambertian scene using multiple images. Although we consider only Lambertian surfaces in this paper, the proposed method, which is global and completely model based, is the first and unavoidable stage for reaching a shape and reflectance estimation method for non-Lambertian surfaces. Basically, our method is a multiview stereo/shape from shading algorithm which allows to recover 3D shapes from Lambertian shading with known illumination conditions. Contrary to previous works that deal with a single material object of the constant albedo, our method works for surfaces with non-constant reflectance parameters, in particular with non-constant albedo. In addition, our algorithm is not based on two or more separate steps – shape and reflectance are jointly recovered in a same process. We verified the proposed method using synthetic images. We will extend our method for non-Lambertian surfaces to improve the robustness to non-Lambertian effects.
