Peter N. Belhumeur

Peter N. Belhumeur
Columbia University | CU · Department of Computer Science

Ph.D., Harvard University

About

155
Publications
45,909
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
38,975
Citations
Citations since 2016
0 Research Items
12023 Citations
201620172018201920202021202205001,0001,5002,000
201620172018201920202021202205001,0001,5002,000
201620172018201920202021202205001,0001,5002,000
201620172018201920202021202205001,0001,5002,000

Publications

Publications (155)
Article
Full-text available
Exemplar-based models have achieved great success on localizing the parts of semi-rigid objects. However, their efficacy on highly articulated objects such as humans is yet to be explored. Inspired by hierarchical object representation and recent application of Deep Convolutional Neural Networks (DCNNs) on human pose estimation, we propose a novel...
Conference Paper
In this paper, we propose a novel part-pair representation for part localization. In this representation, an object is treated as a collection of part pairs to model its shape and appearance. By changing the set of pairs to be used, we are able to impose either stronger or weaker geometric constraints on the part configuration. As for the appearanc...
Conference Paper
We address the problem of large-scale fine-grained visual categorization, describing new methods we have used to produce an online field guide to 500 North American bird species. We focus on the challenges raised when such a system is asked to distinguish between highly similar species of birds. First, we introduce 'one-vs-most classifiers.' By eli...
Patent
Full-text available
Methods, systems, and media for swapping faces in images are provided. In some embodiments, a detected face and face data corresponding to an input image is received. A pose bin associated with the detected face is then identified based on the face data. Next, the detected face is aligned to a generic face associated with the pose bin. At least a p...
Article
We present a novel approach to localizing parts in images of human faces. The approach combines the output of local detectors with a nonparametric set of global models for the part locations based on over 1,000 hand-labeled exemplar images. By assuming that the global models generate the part locations as hidden variables, we derive a Bayesian obje...
Conference Paper
How do you tell a blackbird from a crow? There has been great progress toward automatic methods for visual recognition, including fine-grained visual categorization in which the classes to be distinguished are very similar. In a task such as bird species recognition, automatic recognition systems can now exceed the performance of non-experts - most...
Conference Paper
In this paper, we propose a novel approach for bird part localization, targeting fine-grained categories with wide variations in appearance due to different poses (including aspect and orientation) and subcategories. As it is challenging to represent such variations across a large set of diverse samples with tractable parametric models, we turn to...
Patent
Full-text available
Methods, systems, and media for automatically classifying face images are provided. In some embodiments, features of the face image to be classified for an attribute are selected, wherein each of the features corresponds to a different region of the face image and specifies one or more of a type of pixel data to be evaluated for the region, a norma...
Conference Paper
From a set of images in a particular domain, labeled with part locations and class, we present a method to automatically learn a large and diverse set of highly discriminative intermediate features that we call Part-based One-vs.-One Features (POOFs). Each of these features specializes in discrimination between two particular classes based on the a...
Conference Paper
Describable visual attributes are a powerful way to label aspects of an image, and taken together, build a detailed representation of a scene's appearance. Attributes enable highly accurate approaches to a variety of tasks, including object recognition, face recognition and image retrieval. An important consideration not previously addressed in the...
Conference Paper
The terms Biker, Punk, Hipster, Goth or Surfer often spark visual depictions of individuals with very distinct fashion styles. These visually salient styles can provide insight into the social identity of an individual. However, despite its potential usefulness, little work has been done to automatically classify images of people into social catego...
Conference Paper
We propose a novel approach to fine-grained image classification in which instances from different classes share common parts but have wide variation in shape and appearance. We use dog breed identification as a test case to show that extracting corresponding parts improves classification performance. This domain is especially challenging since the...
Conference Paper
We describe the first mobile app for identifying plant species using automatic visual recognition. The system --- called Leafsnap --- identifies tree species from photographs of their leaves. Key to this system are computer vision components for discarding non-leaf images, segmenting the leaf from an untextured background, extracting features repre...
Conference Paper
Background/Question/Methods Both ecologists and citizen scientists need fast, reliable, and easy-to-use field guides for plant identification. Traditional paper-based guides or more recently introduced electronic field guides essentially use the same methodology, which is based on following dichotomous keys or selecting specific information on mo...
Conference Paper
Recent work has shown that visual attributes are a powerful approach for applications such as recognition, image description and retrieval. However, fusing multiple attribute scores - as required during multi-attribute queries or similarity searches - presents a significant challenge. Scores from different attribute classifiers cannot be combined i...
Article
Full-text available
We propose a new method named compressive structured light for recovering inhomogeneous participating media. Whereas conventional structured light methods emit coded light patterns onto the surface of an opaque object to establish correspondence for triangulation, compressive structured light projects patterns into a volume of participating medium...
Article
We propose a method of face verification that takes advantage of a reference set of faces, disjoint by identity from the test faces, labeled with identity and face part locations. The reference set is used in two ways. First, we use it to perform an "identity-preserving" alignment, warping the faces in a way that reduces differences due to pose and...
Article
Full-text available
We introduce the use of describable visual attributes for face verification and image search. Describable visual attributes are labels that can be given to an image to describe its appearance. This paper focuses on images of faces and the attributes used to describe them, although the concepts also apply to other domains. Examples of face attribute...
Article
Full-text available
For identity related problems, descriptive attributes can take the form of any information that helps represent an in- dividual, including age data, describable visual attributes, and contextual data. With a rich set of descriptive at- tributes, it is possible to enhance the base matching accu- racy of a traditional face identification system throu...
Article
Full-text available
Face recognition systems classically recognize people individually. When presented with a group photograph containing multiple people, such systems implicitly assume statistical independence between each detected face. We question this basic assumption and consider instead that there is a dependence between face regions from the same image; after a...
Conference Paper
We present a novel approach to localizing parts in images of human faces. The approach combines the output of local detectors with a non-parametric set of global models for the part locations based on over one thousand hand-labeled exemplar images. By assuming that the global models generate the part locations as hidden variables, we derive a Bayes...
Article
Full-text available
A major limitation of existing projection display sys-tems is that they rely on a high quality screen for pro-jecting images. We believe that relaxing this restriction will make projectors more useful and widely applicable. The fundamental problem with using an arbitrary sur-face for a screen is that the surface is bound to have its own colors and...
Conference Paper
Full-text available
Helmholtz stereovision methods are limited to binocular stereovision or depth maps reconstruction. In this paper, we extend these methods to recover the full 3D shape of the objects of a scene from multiview Helmholtz stereopsis. Thus, we are able to reconstruct the complete three-dimensional shape of objects made of any arbitrary and unknown bidir...
Article
Full-text available
This special issue arises from papers presented at the Workshop on Photometric Analysis For Computer Vision, held on October 14, 2007 in conjunction with the 11th International Conference on Computer Vision Conference in Rio de Janeiro, Brazil. Photometric analysis is a central aspect of computer vision theory and practice. The way an image looks d...
Article
Full-text available
Dirt on camera lenses, and occlusions from thin objects such as fences, are two important types of artifacts in digital imaging systems. These artifacts are not only an annoyance for photographers, but also a hindrance to computer vision and digital forensics. In this paper, we show that both effects can be described by a single image formation mod...
Article
Dirt on camera lenses, and occlusions from thin objects such as fences, are two important types of artifacts in digital imaging systems. These artifacts are not only an annoyance for photographers, but also a hindrance to computer vision and digital forensics. In this paper, we show that both effects can be described by a single image formation mod...
Conference Paper
Full-text available
We present two novel methods for face verification. Our first method - “attribute” classifiers - uses binary classifiers trained to recognize the presence or absence of describable aspects of visual appearance (e.g., gender, race, and age). Our second method - “simile” classifiers - removes the manual labeling required for attribute classification...
Article
We describe a method for plausible interpolation of images, with a wide range of applications like temporal up-sampling for smooth playback of lower frame rate video, smooth view interpolation, and animation of still images. The method is based on the intuitive idea, that a given pixel in the interpolated frames traces out a path in the source imag...
Article
We describe a method for plausible interpolation of images, with a wide range of applications like temporal up-sampling for smooth playback of lower frame rate video, smooth view interpolation, and animation of still images. The method is based on the intuitive idea, that a given pixel in the interpolated frames traces out a path in the source imag...
Article
Full-text available
The sensor network localization problem is one of determining the Euclidean positions of all sensors in a network given knowledge of the Euclidean positions of some, and knowledge of a number of inter-sensor distances. This paper identifies graphical properties which can ensure unique localizability, and further sets of properties which can ensure...
Conference Paper
Full-text available
We describe a working computer vision system that aids in the identification of plant species. A user photographs an isolated leaf on a blank background, and the system extracts the leaf shape and matches it to the shape of leaves of known species. In a few seconds, the system displays the top matching species, along with textual descriptions and a...
Conference Paper
Full-text available
We have created the first image search engine based entirely on faces. Using simple text queries such as "smiling men with blond hair and mustaches," users can search through over 3.1 million faces which have been automatically labeled on the basis of several facial attributes. Faces in our database have been extracted and aligned from images downl...
Article
Full-text available
In this paper, we present a complete system for automatic face replacement in images. Our system uses a large library of face images created automatically by downloading images from the internet, extracting faces using face detection software, and aligning each extracted face to a common coordinate system. This library is constructed off-line, once...
Article
Full-text available
Imaging of objects under variable lighting directions is an important and frequent practice in computer vision, machine vision, and image-based rendering. Methods for such imaging have traditionally used only a single light source per acquired image. They may result in images that are too dark and noisy, e.g., due to the need to avoid saturation of...
Article
Full-text available
We present a system for refocusing images and videos of dynamic scenes using a novel, single-view depth estimation method. Our method for obtaining depth is based on the defocus of a sparse set of dots projected onto the scene. In contrast to other active illumination techniques, the projected pattern of dots can be removed from each captured image...
Article
Full-text available
Blockwise or Clustered Principal Component Analysis (CPCA) is commonly used to achieve real-time rendering of shadows and glossy reflections with precomputed radiance transfer (PRT). The vertices or pixels are partitioned into smaller coherent regions, and light transport in each region is approximated by a locally low-dimensional subspace using PC...
Article
Full-text available
The properties of virtually all real-world materials change with time, causing their bidirectional reflectance distribution functions (BRDFs) to be time varying. However, none of the existing BRDF models and databases take time variation into consideration; they represent the appearance of a material at a single time instance. In this paper, we add...
Conference Paper
Full-text available
In this paper, we consider using angle of arrival information (bearing) for sensor network localization. The essential property we require in this paper is that a node can infer heading information from its neighbors. We address the uniqueness of network localization solutions by the theory of globally rigid graphs. We show that while the parallel...
Article
Full-text available
The shading in a scene depends on a combination of many factors— how the lighting varies spatially across a surface, how it varies along different directions, the geometric curvature and reflectance properties of objects, and the locations of soft shadows. In this paper, we conduct a complete first order or gradient analysis of lighting, shading an...
Conference Paper
Full-text available
Rendering of clean transparent objects has been well studied in computer graphics. However, real-world transpar- ent objects are seldom clean—their surfaces have a variety of contaminants such as dust, dirt, and lipids. These contaminants produce a number of complex volumetric scattering effects that must be taken into account when creating photore...
Article
Full-text available
In this paper, we provide a theoretical foundation for the problem of network localization in which some nodes know their locations and other nodes determine their locations by measuring the distances to their neighbors. We construct grounded graphs to model network localization and apply graph rigidity theory to test the conditions for unique loca...
Article
Full-text available
Three-dimensional appearance models consisting of spatially varying reflectance functions defined on a known shape can be used in analysis-by-synthesis approaches to a number of visual tasks. The construction of these models requires the measurement of reflectance, and the problem of recovering spatially varying reflectance from images of known sha...
Article
Full-text available
We describe an ongoing project to digitize information about plant specimens and make it available to botanists in the field. This first requires digital images and models, and then effective retrieval and mobile computing mechanisms for accessing this information. We have almost completed a digital archive of the collection of type specimens at th...
Article
Full-text available
For computer graphics rendering, we generally assume that the appearance of surfaces remains static over time. Yet, there are a number of natural processes that cause surface appearance to vary dramatically, such as burning of wood, wetting and drying of rock and fabric, decay of fruit skins, and corrosion and rusting of steel and copper. In this p...
Conference Paper
Full-text available
We present a unied framework for separating specular and diffuse reection components in images and videos of textured scenes. This can be used for specularity removal and for independently processing, ltering, and recom- bining the two components. Beginning with a partial separation provided by an illumination-dependent color space, the challenge i...
Conference Paper
Full-text available
Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric e...
Chapter
Full-text available
We develop a face recognition algorithm which is insensitive to gross variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face under varying illumination di...
Article
Full-text available
In this paper, we provide a theoretical foundation for the problem of network localization in which some nodes know their locations and other nodes determine their locations by measuring the distances to their neighbors. We construct grounded graphs to model network localization and apply graph rigidity theory to test the conditions for unique loca...
Article
Full-text available
Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric e...
Article
Full-text available
Dreams in High Fidelity is a painting that evolves. It was designed and rendered with the Electric Sheep screen-saver, a cyborg mind composed of 30,000 computers and people mediated by a genetic algorithm. Physically it consists of a small computer ...
Conference Paper
Full-text available
This paper is concerned with rigid formations of mobile autonomous agents that have leader-follower architecture. In a previous paper, Baillieul and Suri gave a proposition as a necessary condition for stable rigidity. They also gave a separate theorem as a sufficient condition for stable rigidity. This paper suggests an approach to analyze rigid f...
Article
Full-text available
Cast shadows can be significant in many computer vision applications, such as lighting-insensitive recognition and surface reconstruction. Nevertheless, most algorithms neglect them, primarily because they involve nonlocal interactions in nonconvex regions, making formal analysis difficult. However, many real instances map closely to canonical conf...
Conference Paper
Full-text available
When the shape of an object is known, its appearance is determined by the spatially-varying reectance function dened on its surface. Image-based rendering methods that use geometry seek to estimate this function from image data. Most existing methods recover a unique angular reectance function (e.g., BRDF) at each surface point and provide reectanc...
Conference Paper
Full-text available
We present a photometric stereo method for non-diffuse materials that does not require an explicit reflectance model or reference object. By computing a data-dependent rotation of RGB color space, we show that the specular reflection effects can be separated from the much simpler, diffuse (approximately Lambertian) reflection effects for surfaces t...