Holly E. Rushmeier

• PhD
• Professor (Full) at Yale University

Derived from the mural drawings in the UNESCO-listed Mogao Caves, Dunhuang dance has unique cultural value but faces challenges of digitization and preservation. In this paper, we introduce the first open comprehensive motion capture dataset of Dunhuang dance, Chang-E , including full-body movements documented across eight categories, totaling 40 m...

We present \textsc{Perm}, a learned parametric model of human 3D hair designed to facilitate various hair-related applications. Unlike previous work that jointly models the global hair shape and local strand details, we propose to disentangle them using a PCA-based strand representation in the frequency domain, thereby allowing more precise editing...

Forest fires pose a significant threat worldwide, with Algeria being no exception. In 2020 alone, Algeria witnessed devastating forest fires, affecting over 16,000 hectares of land, a phenomenon largely attributed to the impacts of climate change. Understanding the severity of these fires is crucial for effective management and mitigation efforts....

Despite the ubiquitous use of materials maps in modern rendering pipelines, their editing and control remains a challenge. In this paper, we present an example‐based material control method to augment input material maps based on user‐provided material photos. We train a tileable version of MaterialGAN and leverage its material prior to guide the a...

Graph-based procedural materials are ubiquitous in content production industries. Procedural models allow the creation of photorealistic materials with parametric control for flexible editing of appearance. However, designing a specific material is a time-consuming process in terms of building a model and fine-tuning parameters. Previous work [Hu e...

Despite the ubiquitousness of materials maps in modern rendering pipelines, their editing and control remains a challenge. In this paper, we present an example-based material control method to augment input material maps based on user-provided material photos. We train a tileable version of MaterialGAN and leverage its material prior to guide the a...

We present an implicit neural representation to learn the spatio-temporal space of kinematic motions. Unlike previous work that represents motion as discrete sequential samples, we propose to express the vast motion space as a continuous function over time, hence the name Neural Motion Fields (NeMF). Specifically, we use a neural network to learn t...

Procedural modeling is now the de facto standard of material modeling in industry. Procedural models can be edited and are easily extended, unlike pixel-based representations of captured materials. In this article, we present a semi-automatic pipeline for general material proceduralization. Given Spatially Varying Bidirectional Reflectance Distribu...

In this short paper we present work in progress on creating tools to facilitate 3D reconstruction of cultural heritage. We propose of tools to make reconstruction easier – first we fetch linked open data to help organize source materials, next we extract key contours from photographs to speed up reconstruction, and finally we generate video tours o...

Procedural modeling is now the de facto standard of material modeling in industry. Procedural models can be edited and are easily extended, unlike pixel-based representations of captured materials. In this paper, we present a semi-automatic pipeline for general material proceduralization. Given Spatially-Varying Bidirectional Reflectance Distributi...

We introduce an edge-based procedural texture (EBPT), a procedural model for semi-stochastic texture generation. EBPT quickly generates large textures from a small input image. EBPT focuses on edges as the visually salient features extracted from the input image and organizes into groups with clearly established spatial properties. EBPT allows the...

Non-photorealistic rendering (NPR) and image processing algorithms are widely assumed as a proxy for drawing. However, this assumption is not well assessed due to the difficulty in collecting and registering freehand drawings. Alternatively, tracings are easier to collect and register, but there is no quantitative evaluation of tracing as a proxy f...

Understanding and characterizing material appearance based on human perception is challenging because of the highdimensionality and nonlinearity of reflectance data. We refer to the process of identifying specific characteristics of material appearance within the same category as material estimation, in contrast to material categorization which foc...

This study investigates the potential impact of subsurface light transport on gloss perception for the purposes of broadening our understanding of visual appearance in computer graphics applications. Gloss is an important attribute for characterizing material appearance. We hypothesize that subsurface scattering of light impacts the glossiness perc...

Current computer graphics research practices contain racial biases that have resulted in investigations into "skin" and "hair" that focus on the hegemonic visual features of Europeans and East Asians. To broaden our research horizons to encompass all of humanity, we propose a variety of improvements to quantitative measures and qualitative practice...

Relating small-scale structures to large-scale appearance is a key element in material appearance design. Bi-scale material design requires finding small-scale structures that produce a desired large-scale appearance expressed as a macro-scale BRDF. The adjustment of small-scale geometry and reflectances to achieve a desired appearance can become a...

Computer graphics has an application in virtually every area of human activity. In this Special Issue, we focus on applications in art and cultural heritage. For decades now graphics has played a role in the digitization, restoration, conservation, presentation, and communication of our cultural heritage as well as providing new mediums for reflect...

Efficient workforce training is needed in today's world in which technology is continually changing the nature of work. Students need to be prepared to enter the workforce. Employees need to become lifelong learners to stay up-to-date in their work and to adapt when job functions are eliminated. The training needs are across all industries - includ...

Procedural textures are powerful tools that have been used in graphics for decades. In contrast to the alternative exemplar-based texture synthesis techniques, procedural textures provide user control and fast texture generation with low-storage cost and unlimited texture resolution. However, creating procedural models for complex textures requires...

Time-based media are used in applications ranging from demonstrating the operation of home appliances to explaining new scientific discoveries. However, creating effective time-based media is challenging. We introduce a new framework for authoring and consuming time-based media. An author encodes an animation in a printed code and affixes the code...

The practice of recording and digitizing cultural heritage sites is gaining ground among conservation scientists and scholars in architecture, computer science, and related fields. Recently, the location of sites in areas of conflict has highlighted the urgent need for documenting heritage sites for preservation and posterity. This process is not l...

The study of cultural heritage involves many different activities, including visualizing digital data, analyzing information and sharing results. Current technologies focus on providing better tools for data representation and processing, neglecting the importance of analysis and sharing. In this paper, we present a software system, CHER-Ob, which...

Time-based media (videos, synthetic animations, and virtual reality experiences) are used for communication, in applications such as manufacturers explaining the operation of a new appliance to consumers and scientists illustrating the basis of a new conclusion. However, authoring time-based media that are effective and personalized for the viewer...

We introduce the problem of computing a human-perceived softness measure for virtual 3D objects. As the virtual objects do not exist in the real world, we do not directly consider their physical properties but instead compute the human-perceived softness of the geometric shapes. In an initial experiment, we find that humans are highly consistent in...

Many representations and rendering techniques have been proposed for presenting material appearance in computer graphics. One outstanding problem is evaluating their accuracy. In this paper, we propose assessing accuracy by comparing human judgements of material attributes made when viewing a computer graphics rendering to those made when viewing a...

A 3D modeling system with all-inclusive functionality is too demanding for a casual 3D modeler to learn. There has been a shift towards more approachable systems, with easy-to-learn, intuitive interfaces. However, most modeling systems still employ mouse and keyboard interfaces, despite the ubiquity of tablet devices and the benefits of multi-touch...

In this paper we compare three variants of the graph Laplacian smoothing. The first is the standard synchronous implementation, corresponding to multiplication by the graph Laplacian matrix. The second is a voter process inspired asynchronous implementation, assuming that every vertex is equipped with an independent exponential clock. The third is...

There are many different activities in cultural heritage research, such as digitization, data visualization, information analysis, and dissemination. Much effort has been made to develop better technologies for data capture and processing in cultural heritage. However, digital images and models will become less meaningful if there is not an efficie...

The practice of digitizing cultural heritage sites is gaining ground among conservation scientists and scholars in architecture, art history, computer science, and related fields. Recently, the location of such sites in areas of intense conflict has highlighted the urgent need for documenting cultural heritage for the purposes of preservation and p...

The practice of digitizing cultural heritage sites is gaining ground among conservation scientists and scholars in architecture, art history, computer science, and related fields. Recently, the location of such sites in areas of intense conflict has highlighted the urgent need for documenting cultural heritage for the purposes of preservation and p...

Progress in many domains increasingly benefits from our ability to view the systems through a computational lens, i.e., using computational abstractions of the domains; and our ability to acquire, share, integrate, and analyze disparate types of data. These advances would not be possible without the advanced data and computational cyberinfrastructu...

Material design is the process by which artists or designers set the appearance properties of virtual surface to achieve a desiredlook. This process is often conducted in a virtual synthetic environment however, advances in computer vision tracking andinteractive rendering now makes it possible to design materials in augmented reality (AR), rather...

We propose three automatic algorithms for analyzing digitized medieval manuscripts, text block computation, text line segmentation, and special component extraction, by taking advantage of previous clustering algorithms and a template-matching technique. These three methods are completely automatic, so no user intervention or input is required to m...

In Cultural Heritage projects, it is very important to identify and track weathering effects on monuments in order to design and test conservation strategies. Currently, this mapping is manual work performed by experts based on what they observe and their experience. In this paper, we present a workflow to map the weathering effect known as "scalin...

We propose a method to extract special objects in images of medieval books, which generally represent, for example, figures and capital letters. Instead of working on the single-pixel level, we consider superpixels as the basic classification units for improved time efficiency. More specifically, we classify superpixels into different categories/ob...

We introduce the problem of computing a human-perceived softness measure for virtual 3D objects. As the virtual objects do not exist in the real world, we do not directly consider their physical properties but instead compute the human-perceived softness of the geometric shapes. We collect crowdsourced data where humans rank their perception of the...

While the concept of visual saliency has been previously explored in the areas of mesh and image processing, saliency detection also applies to other sensory stimuli. In this paper, we explore the problem of tactile mesh saliency, where we define salient points on a virtual mesh as those that a human is more likely to grasp, press, or touch if the...

Texture synthesis is a well-established area, with many important applications in computer graphics and vision. However, despite their success, synthesis techniques are not used widely in practice because the creation of good exemplars remains challenging and extremely tedious. In this paper, we introduce an unsupervised method for analyzing textur...

We present a new Monte Carlo based approach for the modelling of Bidirectional Scattering-Surface Reflectance Distribution Function (BSSRDF) for accurate rendering of human skin appearance. The variations of both skin tissues structure and the major chromophores are taken into account correspondingly to the different ethnic and age groups. The comp...

Models of both the shape and material properties of physical objects are needed in many computer graphics applications. In many design applications, even if shape is not needed, it is desirable to start with the material properties of existing non-planar objects. We consider the design of a system to capture both shape and appearance of objects. We...

We propose a simple yet efficient steganalytic algorithm for watermarks embedded by two state-of-the-art 3D watermarking algorithms by Cho et al. The main observation is that while in a clean model the means/variances of Cho et al.’s normalized histogram bins are expected to follow a Gaussian distribution, in a marked model their distribution will...

The study of cultural heritage involves many different activities, including digital data visualization, information analysis and sharing results. Current technologies focus on providing better tools for data representation and processing, neglecting the importance of analysis and sharing. In this paper, we present a software system, CHER-Ob, which...

It is known that humans can be insensitive to large changes in illumination. For example, if an object of interest is extracted from one digital photograph and inserted into another, we do not always notice the differences in illumination between the object and its new background. This inability to spot illumination inconsistencies is often the key...

We present a review of recent techniques for performing geometric analysis in cultural heritage (CH) applications. The survey is aimed at researchers in the areas of computer graphics, computer vision and CH computing, as well as to scholars and practitioners in the CH field. The problems considered include shape perception enhancement, restoration...

A system, method and a computer-readable medium for creating texture exemplars from images are provided. The texture exemplars are created by receiving an image containing a plurality of pixels representing a plurality of textures, wherein each texture in the plurality of textures is configured to be selectable by a user, determining a desired text...

The massive digitization of books and manuscripts has converted millions of works that were once only physical into electronic documents. This conversion has made it possible for scholars to study large bodies of work, rather than just individual texts. This has offered new opportunities for scholarship in the humanities. Much previous work on digi...

We consider the design of an inexpensive system for acquiring material models for computer graphics rendering applications in animation, games and conceptual design. To be useful in these applications a system must be able to model a rich range of appearances in a computationally tractable form. The range of appearance of interest in computer graph...

Massive digital acquisition and preservation of deteriorating historical and artistic documents is of particular importance due to their value and fragile condition. The study and browsing of such digital libraries is invaluable for scholars in the Cultural Heritage field, but requires automatic tools for analyzing and indexing these datasets. We p...

We propose a steganalytic algorithm for watermarks embedded by Cho et al.'s mean-based algorithm [1]. The main observation is that while in a clean model the means of Cho et al.'s normalized histogram bins are expected to follow a Gaussian distribution, in a marked model their distribution will be bimodal. The proposed algorithm estimates the numbe...

This special issue features four new areas of expansion in computer graphics. These new approaches and applications come from recent research in human--computer interaction and computer vision. Each of these new areas is a "big thing" that computer graphics practitioners and researchers need to be aware of. Each area has emerged as the result of ad...

We present a review of recent techniques for performing geometric analysis in cultural heritage applications, targeting the broad community of researchers and practitioners in cultural heritage computing. The problems considered include shape perception enhancement, restoration and preservation support, monitoring over time, object interpretation,...

a) Modeling mode (b) Assembly mode (c) 3D model output Multi-touch gestures 3D components Figure 1. An overview of our novel 3D modeling technique, designed for a multi-touch interface to create 3D models on a tablet device. (a) Users fold 2D sheets of paper in the modeling mode, using gesture-based modeling tools such as folding, bending, extendin...

The BTF data structure was a breakthrough for appearance modeling in computer graphics. More research is needed though to make BTFs practical in rendering applications. We present the first systematic study of the effects of Approximate filtering on the appearance of BTFs, by exploring the spatial, angular and temporal domains over a varied set of...

Art conservators now have access to a wide variety of digital imaging techniques to assist in examining and documenting physical works of art. Commonly used techniques include hyperspectral imaging, 3D scanning, and medical computed tomography imaging. However, viewing most of this digital image data frequently requires both specialized software, w...

Memory colors refer to those colors recalled in association with familiar objects. While some previous work introduces this concept to assist digital image enhancement, their basis, i.e., on-screen memory colors, are not appropriately investigated. In addition, the resulting adjustment methods developed are not evaluated from a perceptual view of p...

Texture is an essential feature in modeling the appearance of objects and is instrumental in making virtual objects appear interesting and/or realistic. Unfortunately, obtaining textures is a labor intensive task requiring parameter tuning for procedural methods or careful photography and post-processing for natural images. Many texture synthesis t...

One major shortcoming of existing bi-scale material design systems is the lack of support for inverse design: there is no way to directly edit the large-scale appearance and then rapidly solve for the small-scale details that approximate that look. Prior work is either too slow to provide quick feedback, or limited in the types of small-scale detai...

Directors employ a process called “color grading” to add color styles to feature films. Color grading is used for a number of reasons, such as accentuating a certain emotion or expressing the signature look of a director. We collect a database of feature film clips and label them with tags such as director, emotion, and genre. We then learn a model...

A massive digital acquisition of huge sets of deteriorating historical documents is mandatory due to their value and delicacy. The study and the browsing of such digital libraries is becoming crucial for scholars in the Cultural Heritage field, but it requires automatic tools for analyzing and indexing those dataset items. We present here a layout...

The past 10 years have seen profound changes in visualization algorithms, techniques, methodologies, and applications. These changes are forcing alterations to visualization courses. Unfortunately, outdated course content recommendations, together with profound changes in the underlying technology and methodology, are producing an unstable ground f...

Art conservators now have access to a wide variety of digital imaging techniques to assist in examining and documenting physical works of art. Commonly used techniques include hyperspectral imaging, 3D scanning and medical CT imaging. However, most of the digital image data requires specialized software to view. The software is often associated wit...

Memory colors refer to those colors recalled in association with familiar objects [Hering 1961]. The deficiency with existing research in this area is that a) screen memory colors have not been rigorously established and b) existing studies do not include extensive human judgements when evaluating image edits based on memory colors. We first perfor...

Sophisticated methods for true spectral rendering have been developed in computer graphics to produce highly accurate images. In addition to traditional applications in visualizing appearance, such methods have potential applications in many areas of scientific study. In particular, we are motivated by the application of studying avian vision and a...

Compositing is one of the most commonly performed operations in computer graphics. A realistic composite requires adjusting the appearance of the foreground and background so that they appear compatible; unfortunately, this task is challenging and poorly understood. We use statistical and visual perception experiments to study the realism of image...

A common weathering effect is the appearance of cracks due to material fractures. Previous exemplar-based aging and weathering methods have either reused images or sought to replicate observed patterns exactly. We introduce a new approach to exemplar-based modeling that creates weathered patterns on synthetic objects by matching the statistics of f...

This issue features four articles on material appearance, which is important in many computer graphics applications. Two articles on general methods consider two distinct issues: measurement and efficient rendering. The other articles examine two extremely different types of specialized materials, illustrating the variety of application requirement...

Architecture is design in spatial context. The only current methods for representing context involve designing in a heavyweight computer-aided design system, using a full model of existing buildings and landscape, or sketching on a panoramic photo. The former is too cumbersome; the latter is too restrictive in viewpoint and in the handling of occlu...

We present the first physically-based interactive system to facilitate the appearance design at different scales consistently, through manipulations of both small-scale geometry and materials. The core of our system is a novel reflectance filtering algorithm, which rapidly computes the large-scale appearance from small-scale details, by exploiting...

Architecture is design in spatial context. The only current methods for representing context involve designing in a heavyweight computer-aided design system, using a full model of existing buildings and landscape, or sketching on a panoramic photo. The former is too cumbersome; the latter is too restrictive in viewpoint and in the handling of occlu...

We present the first physically-based interactive system to facilitate the appearance design at different scales consistently, through manipulations of both small-scale geometry and materials. The core of our system is a novel reflectance filtering algorithm, which rapidly computes the large-scale appearance from small-scale details, by exploiting...

Architecture is design in spatial context. The only current methods for representing context involve designing in a heavyweight computer-aided design system, using a full model of existing buildings and landscape, or sketching on a panoramic photo. The former is too cumbersome; the latter is too restrictive in viewpoint and in the handling of occlu...

CASI is a snapshot capable UV-visible spectral imager for measuring bird plumage. Near apochromatic UV-visible optics were designed and built with an MTF for a 4Mpx detector. Wide-spectral bandwidth data from CASI is then presented.

Digital libraries have now become by far the preferred method for storing large image collections, making it increasingly important to improve issues of search and discovery within these databases. Content-based annotations of images, for classification purposes, have been a particularly significant area of research; however, even the best of such...

The appearance of weathering effects on stone is important for creating outdoor scenes in computer graphics. To achieve them, previous research has built upon physical simulation, which, while yielding a degree of realism, is computationally expensive and inapplicable to the situation when the object geometry is unknown. Also, physical simulation r...

The simulation of weathered appearance is essential in the realistic modeling of urban environments. A representative and particularly difficult effect to produce on a large scale is the effect of fluid flow. Changes in appearance due to flow are the result of both the global effect of large-scale shape, and local effects, such as the detailed roug...

Abstract Bidirectional texture functions (BTFs) represent the appearance of complex materials. Three major shortcomings with BTFs are the bulky storage, the difficulty in editing and the lack of efficient rendering methods. To reduce storage, many compression techniques have been applied to BTFs, but the results are difficult to edit. To facilitate...

This special issue explores recent advances in the acquisition and use of camera images in conjunction with computer graphics techniques. It provides examples of the many ways these advances are expanding our capability to author and share visual experiences.

Digital imaging of cultural heritage artifacts has become a standard practice. Typically, standard commercial cameras, often commodity rather than scientific grade cameras, are used for this purpose. Commercial cameras are optimized for plausible visual reproduction of a physical scene with respect to trichromatic human vision. However, visual repr...

We introduce a tool for organizing images and drawings of archaeological sites. The tool is based on a 3D sketching system developed for conceptual architectural design. The sketching system for design allows a user to represent structures as strokes located on 2D canvases situated in 3D space. The design of a consistent 3D structure evolves as the...

Abstract Papercraft models can serve as inexpensive prototypes in shape design applications. However, in making the models some geometric detail is necessarily lost, and artificial creases may be visible, thereby limiting the utility of these models. To compensate for these practical limitations, we introduce the use of printed patterns on papercra...

We consider the representation of densely sampled scanned 3D objects. Scanning physical objects can be an efficient way of obtaining natural looking input for computer graphics image generation. Scanned meshes however require large amounts of storage. For applications such as computer graphics, preserving the look of objects rather than precise mea...

Images of everyday scenes are frequently used as input for texturing 3D models in computer graphics. Such images include both the texture desired and other extraneous information. In our previous work [Lu et al. 2009], we defined dominant texture as a large homogeneous region in an input sample image and proposed an automatic method to detect domin...