Mark Pauly

Mark Pauly
École Polytechnique Fédérale de Lausanne | EPFL · School of Computer and Communication Sciences

PhD

About

115
Publications
44,459
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
12,669
Citations
Additional affiliations
December 2009 - present
École Polytechnique Fédérale de Lausanne
May 2005 - November 2009
ETH Zurich
August 2003 - February 2005

Publications

Publications (115)
Article
We present a new algorithm for real-time hand tracking on commodity depth-sensing devices. Our method does not require a user-specific calibration session, but rather learns the geometry as the user performs live in front of the camera, thus enabling seamless virtual interaction at the consumer level. The key novelty in our approach is an online op...
Conference Paper
Registration algorithms are an essential component of many computer graphics and computer vision systems. With recent technological advances in RGBD sensors (color plus depth), an active area of research is in techniques combining color, geometry, and learnt priors for robust real-time registration. The goal of this course is to introduce the mathe...
Article
Modern systems for real-time hand tracking rely on a combination of discriminative and generative approaches to robustly recover hand poses. Generative approaches require the specification of a geometric model. In this paper, we propose a the use of sphere-meshes as a novel geometric representation for real-time generative hand tracking. How tightl...
Article
We present a computational method for interactive 3D design and rationalization of surfaces via auxetic materials, i.e., flat flexible material that can stretch uniformly up to a certain extent. A key motivation for studying such material is that one can approximate doubly-curved surfaces (such as the sphere) using only flat pieces, making it attra...
Chapter
Full-text available
We present ShapeOp, a robust and extensible geometric modelling paradigm. ShapeOp builds on top of the state-of-the-art physics solver (Bouaziz et al. in ACM Trans Graph 33:154:1–154:11, 2014). We discuss the main theoretical advantages of the underlying solver and how this influences our modelling paradigm. We provide an efficient open-source C++...
Article
Full-text available
A shape grammar defines a procedural shape space containing a variety of models of the same class, e.g. buildings, trees, furniture, airplanes, bikes, etc. We present a framework that enables a user to interactively design a probability density function (pdf) over such a shape space and to sample models according to the designed pdf. First, we prop...
Article
We present a robust method for capturing articulated hand motions in realtime using a single depth camera. Our system is based on a realtime registration process that accurately reconstructs hand poses by fitting a 3D articulated hand model to depth images. We register the hand model using depth, silhouette, and temporal information. To effectively...
Article
The goal of example-based texture synthesis methods is to generate arbitrarily large textures from limited exemplars in order to fit the exact dimensions and resolution required for a specific modeling task. The challenge is to faithfully capture all of the visual characteristics of the exemplar texture, without introducing obvious repetitions or u...
Article
Full-text available
e present a computational approach for designing wire meshes, i.e., freeform surfaces composed of woven wires arranged in a regular grid. To facilitate shape exploration, we map material properties of wire meshes to the geometric model of Chebyshev nets. This abstraction is exploited to build an efficient optimization scheme. While the theory of Ch...
Article
We present a new method for implicit time integration of physical systems. Our approach builds a bridge between nodal Finite Element methods and Position Based Dynamics, leading to a simple, efficient, robust, yet accurate solver that supports many different types of constraints. We propose specially designed energy potentials that can be solved ef...
Article
Full-text available
We present a new algorithm for computational caustic design. Our algorithm solves for the shape of a transparent object such that the refracted light paints a desired caustic image on a receiver screen. We introduce an optimal transport formulation to establish a correspondence between the input geometry and the unknown target shape. A subsequent 3...
Article
Procedural modeling allows for the generation of innumerable variations of models from a parameterized, conditional or stochastic rule set. Due to the abstractness, complexity and stochastic nature of rule sets, it is often very difficult to have an understanding of the diversity of models that a given rule set defines. We address this problem by p...
Article
Many man-made objects, in particular building facades, exhibit dominant structural relations such as symmetry and regularity. When editing these shapes, a common objective is to preserve these relations. However, often there are numerous plausible editing results that all preserve the desired structural relations of the input, creating ambiguity. W...
Article
Repeated structures are ubiquitous in urban facades. Such repetitions lead to ambiguity in establishing correspondences across sets of unordered images. A decoupled structure-from-motion reconstruction followed by symmetry detection often produces errors: outputs are either noisy and incomplete, or even worse, appear to be valid but actually have a...
Article
In architectural design, surface shapes are commonly subject to geometric constraints imposed by material, fabrication or assembly. Rationalization algorithms can convert a freeform design into a form feasible for production, but often require design modifications that might not comply with the design intent. In addition, they only offer limited su...
Article
We present a new method for implicit time integration of physical systems. Our approach builds a bridge between nodal Finite Element methods and Position Based Dynamics, leading to a simple, efficient, robust, yet accurate solver that supports many different types of constraints. We propose specially designed energy potentials that can be solved ef...
Conference Paper
This paper discusses how realtime face tracking and performance-based animation can facilitate new ways of interaction for computer gaming and other online applications. We identify a number of requirements for realtime face capture and animation systems, and show how recent progress in tracking algorithms advances towards satisfying these requirem...
Article
Mechanical figures that mimic human motions continue to entertain us and capture our imagination. Creating such automata requires expertise in motion planning, knowledge of mechanism design, and familiarity with fabrication constraints. Thus, automaton design remains restricted to only a handful of experts. We propose an automatic algorithm that ta...
Article
The concept of symmetry has received significant attention in computer graphics and computer vision research in recent years. Numerous methods have been proposed to find, extract, encode and exploit geometric symmetries and high-level structural information for a wide variety of geometry processing tasks. This report surveys and classifies recent d...
Article
Rigid registration of two geometric data sets is essential in many applications, including robot navigation, surface reconstruction, and shape matching. Most commonly, variants of the Iterative Closest Point (ICP) algorithm are employed for this task. These methods alternate between closest point computations to establish correspondences between tw...
Conference Paper
Image and geometry registration algorithms are essential components of many computer graphics and computer vision systems. With recent technological advances in RGB-D sensors, robust algorithms that combine 2D image and 3D geometry registration have become an active area of research. This course introduces the basics of 2D/3D registration algorithm...
Article
We present a new algorithm for realtime face tracking on commodity RGB-D sensing devices. Our method requires no user-specific training or calibration, or any other form of manual assistance, thus enabling a range of new applications in performance-based facial animation and virtual interaction at the consumer level. The key novelty of our approach...
Conference Paper
Full-text available
Fabrication of LEGO models in large scale requires careful pre-planning to produce constructable and stable models. We propose a system that, starting with a voxelization of a 3D mesh, merges voxels to form larger bricks, and then analyzes and repairs structural problems, finally outputting a set of building instructions. We also present extensions...
Article
Mesh editing under constraints is a challenging task with numerous applications in geometric modeling, industrial design, and architectural form finding. Recent methods support constraint-based exploration of meshes with fixed connectivity, but commonly lack local control. Because constraints are often globally coupled, a local modification by the...
Article
We propose a computational design approach to generate 3D models composed of interlocking planar pieces. We show how intricate 3D forms can be created by sliding the pieces into each other along straight slits, leading to a simple construction that does not require glue, screws, or other means of support. To facilitate the design process, we presen...
Patent
Systems and methods for enhancing symmetry in 2D and 3D objects are disclosed. At least some embodiments include a computer-readable storage medium including software (executable on a processor) to symmetrize a modeled physical object that causes the processor to identify a plurality of clusters (each including a plurality of symmetric point pairs...
Chapter
Full-text available
Caustics are captivating light patterns created by materials bundling or diverting light by refraction or reflection. We know caustics as random side effects, appearing, for example, at the bottom of a swimming pool. In this paper we show that it is possible to control caustic patterns to form almost any desired shape by optimizing the geometry of...
Article
Fabrication of LEGO models in large scale requires careful pre-planning to produce constructable and stable models. We propose a system that, starting with a voxelization of a 3D mesh, merges voxels to form larger bricks, and then analyzes and repairs structural problems, finally outputting a set of building instructions. We also present extensions...
Article
Consider a dense sampling S of the smooth boundary of a planar shape O, i.e., an open subset of R2R2. We show that the medial axis of the union of Voronoi balls centered at Voronoi vertices inside O has a particularly simple structure: it is the union of all Voronoi vertices inside O and the Voronoi edges connecting them. Therefore, the medial axis...
Article
We introduce a unified optimization framework for geometry processing based on shape constraints. These constraints preserve or prescribe the shape of subsets of the points of a geometric data set, such as polygons, one-ring cells, volume elements, or feature curves. Our method is based on two key concepts: a shape proximity function and shape proj...
Conference Paper
Reconstructing realistic 3D hair geometry is challenging due to omnipresent occlusions, complex discontinuities and specular appearance. To address these challenges, we propose a multi-view hair reconstruction algorithm based on orientation fields with structure-aware aggregation. Our key insight is that while hair's color appearance is view-depend...
Article
Full-text available
We present a novel shape completion technique for creating temporally coherent watertight surfaces from real-time captured dynamic performances. Because of occlusions and low surface albedo, scanned mesh sequences typically exhibit large holes that persist over extended periods of time. Most conventional dynamic shape reconstruction techniques rely...
Article
We introduce a novel framework for image-based 3D reconstruction of urban buildings based on symmetry priors. Starting from image-level edges, we generate a sparse and approximate set of consistent 3D lines. These lines are then used to simultaneously detect symmetric line arrangements while refining the estimated 3D model. Operating both on 2D ima...
Chapter
Full-text available
We present a method for the design of 3D constructions from planar pieces that can be cut easily and cheaply with laser cutters and similar Computer Numerical Control (CNC) machines. By cutting tight slits in intersecting pieces, they can be slid into each other forming stable configurations without any gluing or additional connectors. These constr...
Article
In this demo we present our system for performance-based character animation that enables any user to control the facial expressions of a digital avatar in realtime. Compared to existing technologies, our system is easy to deploy and does not require any face markers, intrusive lighting, or complex scanning hardware. Instead, the user is recorded i...
Article
Full-text available
This paper presents a system for performance-based character animation that enables any user to control the facial expressions of a digital avatar in realtime. The user is recorded in a natural environment using a non-intrusive, commercially available 3D sensor. The simplicity of this acquisition device comes at the cost of high noise levels in the...
Conference Paper
Full-text available
Many tasks in graphical user interfaces require users to interact with elements at various levels of precision. We present FingerGlass, a bimanual technique designed to improve the precision of graphical tasks on multitouch screens. It enables users to quickly navigate to different locations and across multiple scales of a scene using a single hand...
Chapter
Paneling an architectural freeform surface refers to an approximation of the design surface by a set of panels that can be manufactured using a selected technology at a reasonable cost, while respecting the design intent and achieving the desired aesthetic quality of panel layout and surface smoothness. Eigensatz and co-workers [Eigensatz et al. 20...
Technical Report
The realistic reconstruction of hair motion is challenging because of hair's complex occlusion, lack of a well-defined surface, and non-Lambertian material. We present a system for passive capture of dynamic hair performances using a set of high-speed video cameras. Our key insight is that, while hair color is unlikely to match across multiple view...
Article
Full-text available
The emergence of large-scale freeform shapes in architecture poses big challenges to the fabrication of such structures. A key problem is the approximation of the design surface by a union of patches, so-called panels, that can be manufactured with a selected technology at reasonable cost, while meeting the design intent and achieving the desired a...
Article
This paper addresses the fundamental problem of computing stable medial representations of 3D shapes. We propose a spatially adaptive classification of geometric features that yields a robust algorithm for generating medial representations at different levels of abstraction. The recently introduced continuous scale axis transform serves as the math...
Article
We introduce a method for generating facial blendshape rigs from a set of example poses of a CG character. Our system transfers controller semantics and expression dynamics from a generic template to the target blendshape model, while solving for an optimal reproduction of the training poses. This enables a scalable design process, where the user c...
Book
Geometry lies at the core of the architectural design process. It is omnipresent, from the initial determination of form to the final construction. Modern geometric computing provides a variety of tools for the efficient design, analysis, and manufacturing of complex shapes. On the one hand this opens up new horizons for architecture. On the other...
Article
Full-text available
We present a framework and algorithms for robust geometry and motion reconstruction of complex deforming shapes. Our method makes use of a smooth template that provides a crude approximation of the scanned object and serves as a geometric and topological prior for reconstruction. Large-scale motion of the acquired object is recovered using a novel...
Article
Full-text available
"To them, I said, the truth would be literally nothing but the shadows of the images." - Plato, The Republic Shadow art is a unique form of sculptural art where the 2D shadows cast by a 3D sculpture are essential for the artistic effect. We introduce computational tools for the creation of shadow art and propose a design process where the user can...
Article
We propose a new fluid control technique that uses scale-dependent force control to preserve small-scale fluid detail. Control particles define local force fields and can be generated automatically from either a physical simulation or a sequence of target shapes. We use a multi-scale decomposition of the velocity field and apply control forces only...
Article
We present a geometry processing framework that allows direct manipulation or preservation of positional, metric, and curvature constraints anywhere on the surface of a geometric model. Target values for these properties can be specified point-wise or as integrated quantities over curves and surface patches embedded in the shape. For example, the u...
Article
Full-text available
We demonstrate how the scale axis transform can be used to compute a parameterized family of shape skeletons. The skeletons gradually represent only the most important features of a shape, in a scale-adaptive manner. Here a shape O is any bounded open subset of the plane R<sup>2</sup>. The scale axis for scale value s is the medial axis of the mult...
Article
Full-text available
We introduce the scale axis transform, a new skeletal shape representation for bounded open sets O ⊂ &Rdbl;<sup>d</sup>. The scale axis transform induces a family of skeletons that captures the important features of a shape in a scale-adaptive way and yields a hierarchy of successively simplified skeletons. Its definition is based on the medial axi...
Article
Full-text available
We present a complete integrated system for live facial puppetry that enables high-resolution real-time facial expression tracking with transfer to another person's face. The system utilizes a real-time structured light scanner that provides dense 3D data and texture. A generic template mesh, fitted to a rigid reconstruction of the actor's face, is...
Article
Tourist maps are essential resources for visitors to an unfamiliar city because they visually highlight landmarks and other points of interest. Yet, hand-designed maps are static representations that cannot adapt to the needs and tastes of the individual tourist. In this paper we present an automated system for designing tourist maps that selects a...
Article
Full-text available
We introduce a computational framework for discovering regular or repeated geometric structures in 3D shapes. We describe and classify possible regular structures and present an effective algorithm for detecting such repeated geometric patterns in point- or mesh-based models. Our method assumes no prior knowledge of the geometry or spatial location...
Article
We present a registration algorithm for pairs of deforming and partial range scans that addresses the challenges of non-rigid registration within a single non-linear optimization. Our algorithm simultaneously solves for correspondences between points on source and target scans, confidence weights that measure the reliability of each correspondence...
Article
We propose a framework for 3D geometry processing that provides direct access to surface curvature to facilitate advanced shape editing, filtering, and synthesis algorithms. The central idea is to map a given surface to the curvature domain by evaluating its principle curvatures, apply filtering and editing operations to the curvature distribution,...
Article
Full-text available
Symmetry and regularity abound in architectural models, often as a result of economical, manufacturing, functional, or aesthetic con- siderations. We show how recent work on symmetry detection and structure discovery can be utilized to analyze architectural designs and real-world artifacts digitized using D scanning technology. This allows reverse...
Article
We propose a new adaptive space deformation method for interactive shape modeling. A novel energy formulation based on elastically coupled volumetric cells yields intuitive detail preservation even under large deformations. By enforcing rigidity of the cells, we obtain an extremely robust numerical solver for the resulting nonlinear optimization pr...
Article
Full-text available
In the last years triangle meshes have become increasingly popular and are nowadays intensively used in many different areas of computer graphics and geometry processing. In classical CAGD irregular triangle meshes developed into a valuable alternative to traditional spline surfaces, since their conceptual simplicity allows for more flexible and hi...
Article
Full-text available
We present novel adaptive sampling algorithms for particle-based fluid simulation. We introduce a sampling condition based on geometric local feature size that allows focusing computational resources in geometrically complex regions, while reducing the number of particles deep inside the fluid or near thick flat surfaces. Further performance gains...
Article
Full-text available
We present a symmetrization algorithm for geometric objects. Our algorithm enhances approximate symmetries of a model while minimally altering its shape. Symmetrizing deformations are formulated as an optimization process that couples the spatial domain with a transformation configuration space, where symmetries can be expressed more naturally and...
Article
Full-text available
We present an algorithm that generates natural and intuitive deformations via direct manipulation for a wide range of shape representations and editing scenarios. Our method builds a space deformation represented by a collection of affine transformations organized in a graph structure. One transformation is associated with each graph node and appli...
Conference Paper
Full-text available
We present an example-based surface reconstruction method for scanned point sets. Our approach uses a database of local shape priors built from a set of given context models that are chosen specifically to match a specific scan. Local neighborhoods of the input scan are matched with enriched patches of these models at multiple scales. Hence, instea...
Article
Full-text available
Given a dense sampling S of the smooth boundary of a planar shape O. We show that the medial axis of the union of Voronoi balls centered at Voronoi vertices inside O has a particularly simple structure and can be computed more eciently and robustly than for a general union of balls.
Article
Full-text available
We illustrate a simple algorithm for approximating the medial axis of a 2D shape with smooth boundary from a sample of this boundary. The algorithm is compared to a more general approximation method that builds on the same idea, namely, to approximate the shape by a union of balls. While not as general, our algorithm is simpler, faster and numerica...
Article
Full-text available
Conformal alpha shapes are a new filtration of the Delaunay triangulation of a finite set of points in ℝd. In contrast to (ordinary) alpha shapes the new filtration is parameterized by a local scale parameter instead of the global scale parameter in alpha shapes. The local scale parameter conforms to the local geometry and is motivated from applica...