Robert Pless

• Doctor of Philosophy
• Chair at George Washington University

Composed image retrieval (CIR) enables users to search images using a reference image combined with textual modifications. Recent advances in vision-language models have improved CIR, but dataset limitations remain a barrier. Existing datasets often rely on simplistic, ambiguous, or insufficient manual annotations, hindering fine-grained retrieval....

We explore the use of deep convolutional neural networks (CNNs) trained on overhead imagery of biomass sorghum to ascertain the relationship between single nucleotide polymorphisms (SNPs), or groups of related SNPs, and the phenotypes they control. We consider both CNNs trained explicitly on the classification task of predicting whether an image sh...

Pair-wise loss is an approach to metric learning that learns a semantic embedding by optimizing a loss function that encourages images from the same semantic class to be mapped closer than images from different classes. The literature reports a large and growing set of variations of the pair-wise loss strategies. Here we decompose the gradient of t...

We introduce a simple approach to understanding the relationship between single nucleotide polymorphisms (SNPs), or groups of related SNPs, and the phenotypes they control. The pipeline involves training deep convolutional neural networks (CNNs) to differentiate between images of plants with reference and alternate versions of various SNPs, and the...

Triplet loss is an extremely common approach to distance metric learning. Representations of images from the same class are optimized to be mapped closer together in an embedding space than representations of images from different classes. Much work on triplet losses focuses on selecting the most useful triplets of images to consider, with strategi...

Triplet loss is an extremely common approach to distance metric learning. Representations of images from the same class are optimized to be mapped closer together in an embedding space than representations of images from different classes. Much work on triplet losses focuses on selecting the most useful triplets of images to consider, with strategi...

Estimating strain on surfaces of deforming three dimensional (3D) structures is a critical need in experimental mechanics. Although single-camera techniques excel at estimating deformation on a surface parallel to the imaging plane, they are prone to artifact for 3D motion because they cannot distinguish between out-of-plane motion and in-plane dil...

Background: Measurement of plant traits with precision and speed on large populations has emerged as a critical bottleneck in connecting genotype to phenotype in genetics and breeding. This bottleneck limits advancements in understanding plant genomes and the development of improved, high-yielding crop varieties. Results: Here we demonstrate the...

Investigations of sex trafficking sometimes have access to photographs of victims in hotel rooms. These images directly link victims to places, which can help verify where victims have been trafficked or where traffickers might operate in the future. Current machine learning approaches give promising results in image search to find the matching hot...

Deep metric learning is often used to learn an embedding function that captures the semantic differences within a dataset. A key factor in many problem domains is how this embedding generalizes to new classes of data. In observing many triplet selection strategies for Metric Learning, we find that the best performance consistently arises from appro...

We propose to implicitly learn to extract geo-temporal image features, which are mid-level features related to when and where an image was captured, by explicitly optimizing for a set of location and time estimation tasks. To train our method, we take advantage of a large image dataset, captured by outdoor webcams and cell phones. The only form of...

Recognizing a hotel from an image of a hotel room is important for human trafficking investigations. Images directly link victims to places and can help verify where victims have been trafficked, and where their traffickers might move them or others in the future. Recognizing the hotel from images is challenging because of low image quality, uncomm...

Deep metric learning seeks to define an embedding where semantically similar images are embedded to nearby locations, and semantically dissimilar images are embedded to distant locations. Substantial work has focused on loss functions and strategies to learn these embeddings by pushing images from the same class as close together in the embedding s...

Recognizing a hotel from an image of a hotel room is important for human trafficking investigations. Images directly link victims to places and can help verify where victims have been trafficked, and where their traffickers might move them or others in the future. Recognizing the hotel from images is challenging because of low image quality, uncomm...

Background: Precise measurement of plant traits with precision and speed on large populations has emerged as a critical bottleneck in connecting genotype to phenotype in genetics and breeding. This bottleneck limits advancements in understanding plant genomes and the development of improved, high-yielding crop varieties. Results: Here we demonstrat...

For convolutional neural network models that optimize an image embedding, we propose a method to highlight the regions of images that contribute most to pairwise similarity. This work is a corollary to the visualization tools developed for classification networks, but applicable to the problem domains better suited to similarity learning. The visua...

Quantifying dynamic strain fields from time-resolved volumetric medical imaging and microscopy stacks is a pressing need for radiology and mechanobiology. A critical limitation of all existing techniques is regularization: because these volumetric images are inherently noisy, current strain mapping techniques must impose either displacement regular...

Learning embedding functions, which map semantically related inputs to nearby locations in a feature space supports a variety of classification and information retrieval tasks. In this work, we propose a novel, generalizable and fast method to define a family of embedding functions that can be used as an ensemble to give improved results. Each embe...

Learning embedding functions, which map semantically related inputs to nearby locations in a feature space supports a variety of classification and information retrieval tasks. In this work, we propose a novel, generalizable and fast method to define a family of embedding functions that can be used as an ensemble to give improved results. Each embe...

Publicly available, outdoor webcams continuously view the world and share images. These cameras include traffic cams, campus cams, ski-resort cams, etc. The Archive of Many Outdoor Scenes (AMOS) is a project aiming to geolocate, annotate, archive, and visualize these cameras and images to serve as a resource for a wide variety of scientific applica...

We propose Deep Feature Interpolation (DFI), a new data-driven baseline for automatic high-resolution image transformation. As the name suggests, it relies only on simple linear interpolation of deep convolutional features from pre-trained convnets. We show that despite its simplicity, DFI can perform high-level semantic transformations like "make...

We design and demonstrate a passive physical object whose appearance changes to give a discrete encoding of its pose. This object is created with a microlens array that is placed on top of a black and white pattern; when viewed from a particular viewpoint, the lenses appear black or white depending on the part of the pattern that each microlens pro...

Introduction Active transportation opportunities and infrastructure are an important component of a community’s design, livability, and health. Features of the built environment influence active transportation, but objective study of the natural experiment effects of built environment improvements on active transportation is challenging. The purpos...

Significance Cancer patients undergoing chemotherapy experience high rates of dose-limiting morbidity. Recently, short-term fasting prior to chemotherapy was shown to decrease toxicity. Herein we report that fasting protects multiple small intestinal stem cell populations marked by Lgr5 , Bmi1 , or HopX expression and maintains barrier function to...

Active transportation is an important contributor to physical activity. Understanding active transportation trends and transportation mode share is important to public health research and city planners. Objective measurement of active transportation can be costly and time-consuming, and existing camera-based algorithms, while developing, are functi...

Images are important to fighting sex trafficking because they are: (a) used to advertise for sex services, (b) shared among criminal networks, and (c) connect a person in an image to the place where the image was taken. This work explores the ability to link images to indoor places in order to support the investigation and prosecution of criminal a...

Understanding the pose of an object is fundamental to a variety of visual tasks, from trajectory estimation of UAVs to object tracking for augmented reality. Fiducial markers are visual targets designed to simplify this process by being easy to detect, recognize, and track. They are often based on features that are partially invariant to lighting,...

Short-term fasting in mice has been shown to increase survival from lethal doses of chemotherapy; however, the route of protection in these animals is unknown. In this study we demonstrate that fasting prior to chemotherapy protects small intestinal (SI) stem cells in mice exposed to high dose etoposide. Histologic and in vitro crypt culture analys...

ViewRay's MRIdian system acquires cine-MR images during treatment, allowing for real-time visualization of the tumor without additional dose. We propose a novel method for tracking the tumor target on these cine-MR images in real-time. 9 patients with mobile abdominal tumors (3 stomach, 1 pancreas, 1 adrenal, 2 mesenteric, 2 peritoneal) were imaged...

Many vision and augmented reality applications require knowing the rotation of the camera relative to an object or scene. In this paper we propose to create a structured light field designed explicitly to simplify the estimation of camera rotation. The light field is created using a lenticular sheet with a color coded backplane pattern, creating a...

Many computer vision applications rely on matching features of a query image to reference data sets, but little work has explored how quickly data sets become out of date. In this paper we measure feature matching performance across 5 years of time-lapse data from 20 static cameras to empirically study how feature matching is affected by changing s...

Five years ago we reported at AIPR on a nascent project to archive images from every webcam in the world and to develop algorithms to geo-locate, calibrate, and annotate this data. This archive of many outdoor scenes (AMOS) has now grown to include 28000 live outdoor cameras and over 630 million images. This is actively being used in projects rangi...

The vast amount of public photographic data posted and shared on Facebook, Instragram and other forms of social media offers an unprecedented visual archive of the world. This archive captures events ranging from birthdays, trips, and graduations to lethal conflicts and human rights violations. Because this data is public, it has led to a new genre...

When mechanical factors underlie growth, development, disease or healing, they often function through local regions of tissue where deformation is highly concentrated. Current optical techniques to estimate deformation can lack precision and accuracy in such regions due to challenges in distinguishing a region of concentrated deformation from an er...

There are tens of thousands of publicly available webcams which constantly view the world and share those images. These cameras include traffic cams, campus cams, ski-resort cams, etc. The Archive of Many Outdoor Scenes (AMOS) is a project that aims to geo-locate, calibrate, annotate, archive and visualize these cameras to serve as an imaging resou...

Image processing techniques are used to understand the local specimen behavior in tests of large-scale civil structures. One such technique is Digital Image Correlation (DIC), which monitors the motion of a speckle pattern on the surface of a specimen. This paper introduces the Motion Component Analysis (MCA) method, which is based on (a) a princip...

In outdoor images, cast shadows define 3D constraints between the sun, the points casting a shadow, and the surfaces onto which shadows are cast. This cast shadow structure provides a powerful cue for 3D reconstruction, but requires that shadows be tracked over time, and this is difficult as shadows have minimal texture. Thus, we develop a shadow t...

Significance How different cortical regions are coordinated during a cognitive task is fundamentally important to understanding brain function. At rest, the brain is subdivided into different functional networks that are bound together at very slow oscillating time scales. Less is understood about how this networked behavior operates during the bri...

A method of detecting optical defects in a transparency may comprise the steps of providing a digital image of the transparency having a plurality of image pixels and detecting at least one candidate defect. The candidate defect may be detected by determining a grayscale intensity of each one of the image pixels and calculating an intensity gradien...

The Archive of Many Outdoor Scenes has captured 400 million images. Many of these cameras and images are of street intersections, a subset of which has experienced built environment improvements during the past seven years. We identified six cameras in Washington, DC, and uploaded 120 images from each before a built environment change (2007) and af...

Introduction: Fewer than 50% of adults meet CDC guidelines for physical activity (PA). The built environment (BE) is a culprit for limited PA. The study objective was to analyze existing, online public data feeds to quantify effectiveness of BE interventions and examine the impact of seasonality on these interventions. The Archive of Many Outdoor S...

We describe algorithms that use cloud shadows as a form of stochastically structured light to support 3D scene geometry estimation. Taking video captured from a static outdoor camera as input, we use the relationship of the time series of intensity values between pairs of pixels as the primary input to our algorithms. We describe two cues that rela...

Rephotography is the process of capturing the same scene at a different time, in order to capture changes. Previous work at SIGGRAPH [BAE2010] demonstrated the ability for smart-phone apps to guide a user to the correct viewpoint, here we promote the use of such tools distributed widely over space and time, by enabling collaborative projects that a...

Mechanical characterization of inhomogeneous and/or geometrically complex biological tissues requires precise and accurate determination of strain fields. Digital image correlation is a well established technique for determining strain fields on the surfaces of deforming materials. The technique involves matching patterns between pairs of images to...

Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episo...

Recovering shadows is an important step for many vision algorithms. Current approaches that work with time-lapse sequences are limited to simple thresholding heuristics. We show these approaches only work with very careful tuning of parameters, and do not work well for long-term time-lapse sequences taken over the span of many months. We introduce...

The ecological sciences face the challenge of making measurements to detect subtle changes sometimes over large areas across varied temporal scales. The challenge is thus to measure patterns of slow, subtle change occurring along multiple spatial and temporal scales, and then to visualize those changes in a way that makes important variations visce...

A global network of webcams offers unique viewpoints from tens of thousands of locations. Understanding the geographic context of this imagery is vital in using these cameras for quantitative environmental monitoring or surveillance applications. We derive robust geo-calibration constraints that allow users to geo-register static or pan-tilt-zoom c...

We consider the problem of estimating the current satellite cloud map from a collection of broadly distributed, ground-based webcams. The approach uses historical, geo-referenced satellite imagery to learn a mapping between the satellite image and the ground imagery. We explore representational choices for inferring the cloud status based on the gr...

Thirty years ago, a young girl was found decapitated. Her identity remains unknown, and neither her head nor her killer have been found. Until recently, the location of her grave was lost, preventing any efforts to identify her using modern forensic techniques. This paper presents a case study on the use of burial photos to accurately and precisely...

Characterizing how cells in three-dimensional (3D) environments or natural tissues respond to biophysical stimuli is a longstanding challenge in biology and tissue engineering. We demonstrate a strategy to monitor morphological and mechanical responses of contractile fibroblasts in a 3D environment. Cells responded to stretch through specific, cell...

Coordinated navigation within tissues is essential for cells of the innate immune system to reach the sites of inflammatory processes, but the signals involved are incompletely understood. Here we demonstrate that NG2(+) pericytes controlled the pattern and efficacy of the interstitial migration of leukocytes in vivo. In response to inflammatory me...

In this work, we present a method to uncover shape from webcams "in the wild." We present a variant of photometric stereo which uses the sun as a distant light source, so that lighting direction can be computed from known GPS and timestamps. We propose an iterative, non-linear optimization process that optimizes the error in reproducing all images...

Crowd-sourcing tools such as Mechanical Turk are popular for annotation of large scale image data sets. Typically, these annotations consist of bounding boxes or coarse outlines of objects, in order to keep the interface as simple as possible and to respect browser constraints. However, as most browsers now contain functionality to quickly process...

We introduce the Longterm Observation of Scenes (with Tracks) dataset. This dataset comprises videos taken from streaming outdoor webcams, capturing the same half hour, each day, for over a year. LOST contains rich metadata, including geolocation, day-by-day weather annotation, object detections, and tracking results. We believe that sharing this d...

The term phenology refers to both the seasonal rhythms of plants and animals, and the study of these rhythms. Plant phenological processes, such as when leaves emerge in the spring and change color in the autumn, are highly responsive to year-to-year variation in weather as well as longer-term changes in climate, particularly as related to temperat...

The PhenoCam website (http://phenocam.unh.edu/) is tasked with acquiring, processing, and archiving web imagery from inexpensive webcams to be used for scientific studies of phenological processes. The project involves two overlapping networks of cameras: AMOS (Archive of Many Outdoor Scenes), archives years of images from over 17,000 cameras, and...

We describe a complete system for quantitatively measuring the optical distortion in aircraft windshields and automatically classifying that distortion as acceptable or not. The system comprises two parts: The first uses digital imaging of a known grid pattern through the windshield of interest to create a distortion map of that windshield; the sec...

We characterize a class of videos consisting of very small but potentially complicated motions. We find that in these scenes, linear appearance variations have a direct relationship to scene motions. We show how to interpret appearance variations captured through a PCA decomposition of the image set as a scene-specific non-parametric motion basis....

Gelatin cylinder strain plots. Lagrangian strain fields for the first four principal components (PCs) of a rotating gelatin cylinder. The polar strain plots resemble the Bessel functions that appear in solutions to analogous problems. (TIF)

Human brain strain plots. Lagrangian strain fields corresponding to the first four principal components (PCs) of a human brain rotating inside of a skull in vivo. (TIF)

Vibrating plate modal coefficients. Modal coefficients for each principal component of the vibrating plate. The inset shows that the majority of variance was due to the first principal component. The noise evident arose because the simulation was discrete and not continuous. (TIF)

Non-destructive measurement of acceleration-induced displacement fields within a closed object is a fundamental challenge. Inferences of how the brain deforms following skull impact have thus relied largely on indirect estimates and course-resolution cadaver studies. We developed a magnetic resonance technique to quantitatively identify the modes o...

The forces exerted on the flagellum of the swimming alga Chlamydomonas reinhardtii by surrounding fluid are estimated from video data. “Wild-type” cells, as well as cells lacking inner dynein arms (ida3) and cells lacking outer dynein arms (oda2) were imaged (350 fps; 125 nm). Digital image registration and sorting algorithms provide high-resolutio...

The distributed propulsive forces exerted on the flagellum of the swimming alga Chlamydomonas reinhardtii by surrounding fluid were estimated from experimental image data. Images of uniflagellate mutant Chlamydomonas cells were obtained at 350 frames/s with 125-nm spatial resolution, and the motion of the cell body and the flagellum were analyzed i...

We consider the problem of geo-locating static cameras from long-term time-lapse imagery. This problem has received significant attention recently, with most methods making strong assumptions on the geometric structure of the scene. We explore a simple, robust cue that relates overall image intensity to the zenith angle of the sun (which need not b...

In surveillance and environmental monitoring applications, it is common to have millions of images of a particular scene. While there exist tools to find particular events, anomalies, human actions and behaviors, there has been little investigation of tools which allow more exploratory searches in the data. This paper proposes modifications to PCA...

Web services supporting deep integration between video data and geographic information systems (GIS) empower a large user base to build on popular tools such as Google Earth and Google Maps. Here we extend web interfaces designed explicitly for novice users to integrate streaming video with 3D GIS, and work to dramatically simplify the task of rete...

Many wireless sensor networks require sufficient sensing coverage over long periods of time. To conserve energy, a coverage maintenance protocol achieves desired coverage by activating only a subset of nodes, while allowing the others to sleep. Existing coverage maintenance protocols are often designed based on simplistic sensing models that do not...

Immune-mediated pulmonary diseases are a significant public health concern. Analysis of leukocyte behavior in the lung is essential for understanding cellular mechanisms that contribute to normal and diseased states. Here, we used two-photon imaging to study neutrophil extravasation from pulmonary vessels and subsequent interstitial migration. We f...

We explore the use of clouds as a form of structured lighting to capture the 3D structure of outdoor scenes observed over time from a static camera. We derive two cues that relate 3D distances to changes in pixel intensity due to clouds shadows. The first cue is primarily spatial, works with low frame-rate time lapses, and supports estimating focal...

Global satellite imagery provides nearly ubiquitous views of the Earth's surface, and the tens of thousands of webcams provide live views from near Earth viewpoints. Combining these into a single application creates live views in the global context, where cars move through intersections, trees sway in the wind, and students walk across campus in re...

The relationship between skull acceleration and brain injury is not well understood, in large part because of the challenge of visualizing the brain’s mechanical response in vivo. This difficulty also complicates the validation of computational mechanics predictions. Our dynamic magnetic resonance (MR) imaging suggests an important role for the att...

This study describes the measurement of fields of relative displacement between the brain and the skull in vivo by tagged magnetic resonance imaging and digital image analysis. Motion of the brain relative to the skull occurs during normal activity, but if the head undergoes high accelerations, the resulting large and rapid deformation of neuronal...

Compressive-sensing cameras are an important new class of sensors that have different design constraints than standard cameras. Surprisingly, little work has explored the relationship between compressive-sensing measurements and differential image motion. We show that, given modest constraints on the measurements and image motions, we can omit the...

The relationship between skull acceleration and brain injury is not well understood, in large part because of the challenge of visualizing the brain’s mechanical response in vivo. This difficulty also complicates the validation of computational mechanics predictions. Our dynamic magnetic resonance (MR) imaging suggests an important role for the att...