About
788
Publications
113,614
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
57,368
Citations
Publications
Publications (788)
Lensless cameras relax the design constraints of traditional cameras by shifting image formation from analog optics to digital post-processing. While new camera designs and applications can be enabled, lensless imaging is very sensitive to unwanted interference (other sources, noise, etc.). In this work, we address a prevalent noise source that has...
Previous work has not shown if learned components for lensless image reconstruction can generalize to physical modifications. In this work, we train a reconstruction approach on an open-sourced, multi-mask dataset, and demonstrate improved generalizability.
We demonstrate a throughput increase of third-harmonic generation microscopy imaging using arbitrary illumination patterns. A ‘camera-in-the-loop’ feedback control optimizes the hologram displayed by a spatial light modulator and two different image reconstruction methods are implemented.
For safe and efficient operation, mobile robots need to perceive their environment, and in particular, perform tasks such as obstacle detection, localization, and mapping. Although robots are often equipped with microphones and speakers, the audio modality is rarely used for these tasks. Compared to the localization of sound sources, for which many...
We propose an LCD-based lensless camera for compressive and programmable sensing. The system is low-cost (less than 100 USD) and reproducible. We model the proposed system, and demonstrate experimental results for imaging and classification.
Lensless imaging can provide visual privacy due to the highly multiplexed characteristic of its measurements. However, this alone is a weak form of security, as various adversarial attacks can be designed to invert the one-to-many scene mapping of such cameras. In this work, we enhance the privacy provided by lensless imaging by (1) downsampling at...
Lippmann (or interferential) photography is the first and only analog photography method that can capture the full color spectrum of a scene in a single take. This technique, invented more than a hundred years ago, records the colors by creating interference patterns inside the photosensitive plate. Lippmann photography provides a great opportunity...
As event-based sensing gains in popularity, theoretical understanding is needed to harness this technology's potential. Instead of recording video by capturing frames, event-based cameras have sensors that emit events when their inputs change, thus encoding information in the timing of events. This creates new challenges in establishing reconstruct...
Lippmann (or interferential) photography is the first and only analog photography method that can capture the full color spectrum of a scene in a single take. This technique, invented more than a hundred years ago, records the colors by creating interference patterns inside the photosensitive plate. Lippmann photography provides a great opportunity...
For safe and efficient operation, mobile robots need to perceive their environment, and in particular, perform tasks such as obstacle detection, localization, and mapping. Although robots are often equipped with microphones and speakers, the audio modality is rarely used for these tasks. Compared to the localization of sound sources, for which many...
In event-based sensing, many sensors independently and asynchronously emit events when there is a change in their input. Event-based sensing can present significant improvements in power efficiency when compared to traditional sampling, because (1) the output is a stream of events where the important information lies in the timing of the events, an...
We consider three-dimensional cubic barcodes, consisting of smaller cubes, each built from one of two possible materials and carry one bit of information. To retrieve the information stored in the barcode, we measure a 2-D projection of the barcode using a penetrating wave such as X-rays, either using parallel-beam or cone-beam scanners from an unk...
In event-based sensing, many sensors independently and asynchronously emit events when there is a change in their input. Event-based sensing can present significant improvements in power efficiency when compared to traditional sampling, because (1) the output is a stream of events where the important information lies in the timing of the events, an...
Significance
Gabriel Lippmann won the 1908 Nobel Prize in Physics for his method of reproducing colors in photography. Despite the significance of this result, there are still misconceptions regarding the approach. We provide a complete end-to-end analysis of the process and show, both theoretically and experimentally, how the spectrum reflected fr...
Finite rate of innovation (FRI) is a powerful reconstruction framework enabling the recovery of sparse Dirac streams from uniform low-pass filtered samples. An extension of this framework, called generalised FRI (genFRI), has been recently proposed for handling cases with arbitrary linear measurement models. In this context, signal reconstruction a...
Finite rate of innovation (FRI) is a powerful reconstruction framework enabling the recovery of sparse Dirac streams from uniform low-pass filtered samples. An extension of this framework, called generalised FRI (genFRI), has been recently proposed for handling cases with arbitrary linear measurement models. In this context, signal reconstruction a...
We study the problem of localizing a configuration of points and planes from the collection of point-to-plane distances. This problem models simultaneous localization and mapping from acoustic echoes as well as the “structure from sound” approach to microphone localization with unknown sources. In our earlier work we proposed computational methods...
Sampling is classically performed by recording the amplitude of an input signal at given time instants; however, sampling and reconstructing a signal using multiple devices in parallel becomes a more difficult problem to solve when the devices have an unknown shift in their clocks.
Matrix (or operator) recovery from linear measurements is a well studied problem. However, there are situations where only bi-linear or quadratic measurements are available. A bi-linear or quadratic problem can be easily transformed to a linear one, but it raises questions when the linearized problem is solvable and what is the cost of linearizatio...
Euclidean distance matrices (EDMs) are a major tool for localization from distances, with applications ranging from protein structure determination to global positioning and manifold learning. They are, however, static objects which serve to localize points from a snapshot of distances. If the objects move, one expects to do better by modeling the...
Conventional sampling focuses on encoding and decoding bandlimited signals by recording signal amplitudes at known time points. Alternately, sampling can be approached using biologically-inspired schemes. Among these are integrate-and-fire time encoding machines (IF-TEMs). They behave like simplified versions of spiking neurons and encode their inp...
We study the accuracy of triangulation in multi-camera systems with respect to the number of cameras. We show that, under certain conditions, the optimal achievable reconstruction error decays quadratically as more cameras are added to the system. Furthermore, we analyse the error decay-rate of major state-of-the-art algorithms with respect to the...
In a variety of fields, in particular those involving imaging and optics, we often measure signals whose phase is missing or has been irremediably distorted. Phase retrieval attempts to recover the phase information of a signal from the magnitude of its Fourier transform to enable the reconstruction of the original signal. Solving the phase retriev...
Sampling is classically performed by recording the amplitude of the input at given time instants; however, sampling and reconstructing a signal using multiple devices in parallel becomes a more difficult problem to solve when the devices have an unknown shift in their clocks. Alternatively, one can record the times at which a signal (or its integra...
We study the problem of localizing a configuration of points and planes from the collection of point-to-plane distances. This problem models simultaneous localization and mapping from acoustic echoes as well as the notable "structure from sound" approach to microphone localization with unknown sources. In our earlier work we proposed computational...
Euclidean distance matrices (EDMs) are a major tool for localization from distances, with applications ranging from protein structure determination to global positioning and manifold learning. They are, however, static objects which serve to localize points from a snapshot of distances. If the objects move, one expects to do better by modeling the...
Traditional sampling results assume that the sample locations are known. Motivated by simultaneous localization and mapping (SLAM) and structure from motion (SfM), we investigate sampling at unknown locations. Without further constraints, the problem is often hopeless. For example, we recently showed that, for polynomial and bandlimited signals, it...
In a variety of fields, in particular those involving imaging and optics, we often measure signals whose phase is missing or has been irremediably distorted. Phase retrieval attempts to recover the phase information of a signal from the magnitude of its Fourier transform to enable the reconstruction of the original signal. Solving the phase retriev...
Estimating Diracs in continuous two or higher dimensions is a fundamental problem in imaging. Previous approaches extended one dimensional methods, like the ones based on finite rate of innovation (FRI) sampling, in a separable manner, e.g., along the horizontal and vertical dimensions separately in 2D. The separate estimation leads to a sample com...
We study the accuracy of triangulation in multi-camera systems with respect to the number of cameras. We show that, under certain conditions, the optimal achievable reconstruction error decays quadratically as more cameras are added to the system. Furthermore, we analyse the error decay-rate of major state-of-the-art algorithms with respect to the...
The connectivity of a neuronal network has a major effect on its functionality and role. It is generally believed that the complex network structure of the brain provides a physiological basis for information processing. Therefore, identifying the network’s topology has received a lot of attentions in neuroscience and has been the center of many re...
We address the problem of sampling and reconstruction of sparse signals with finite rate of innovation. We derive general conditions under which perfect reconstruction is possible for sampling kernels satisfying Strang-Fix conditions. Previous results on the subject consider two particular cases; when the kernel is able to reproduce (complex) expon...
Relightable photographs are alternatives to traditional photographs as they provide a richer viewing experience. However, the complex acquisition systems of existing techniques have restricted its usage to specialized setups. We introduce an easy-to-use and affordable solution for using smartphones to acquire the reflectance of paintings and simila...
Recent sampling results enable the reconstruction of signals composed of streams of fixed-shaped pulses. These results have found applications in topics as varied as channel estimation, biomedical imaging and radio astronomy. However, in many real signals, the pulse shapes vary throughout the signal. In this paper, we show how to sample and perfect...
We show how introducing known scattering can be used in direction of arrival estimation by a single sensor. We first present an analysis of the geometry of the underlying measurement space and show how it enables localizing white sources. Then, we extend the solution to more challenging non-white sources like speech by including a source model and...
The relation between sensor resolution and the optics of a digital camera is determined by the Nyquist sampling theorem: the sampling frequency should be larger than twice the maximum frequency of the image content coming out of the optical system. If a lower resolution is used, the output is aliased. Aliasing in digital images is often considered...
In this paper we present FRIDA---an algorithm for estimating directions of arrival of multiple wideband sound sources. FRIDA combines multi-band information coherently and achieves state-of-the-art resolution at extremely low signal-to-noise ratios. It works for arbitrary array layouts, but unlike the various steered response power and subspace met...
We study simultaneous localization and mapping with a device that uses reflections to measure its distance from walls. Such a device can be realized acoustically with a synchronized collocated source and receiver; it behaves like a bat with no capacity for directional hearing or vocalizing. In this paper we generalize our previous work in 2D, and s...
We study the problem of simultaneously reconstructing a polygonal room and a trajectory of a device equipped with a (nearly) collocated omnidirectional source and receiver. The device measures arrival times of echoes of pulses emitted by the source and picked up by the receiver. No prior knowledge about the device's trajectory is required. Most exi...
Stained glass windows are designed to reveal their powerful artistry under diverse and time-varying lighting conditions; virtual relighting of stained glass, therefore represents an exceptional tool for the appreciation of this age old art form. However, as opposed to most other artifacts, stained glass windows are extremely difficult if not imposs...
Linear inverse problems are ubiquitous. Often the measurements do not follow a Gaussian distribution. Additionally, a model matrix with a large condition number can complicate the problem further by making it ill-posed. In this case, the performance of popular estimators may deteriorate significantly. We have developed a new estimator that is both...
We address the problem of jointly localizing a robot in an unknown room and estimating the room geometry from echoes. Unlike earlier work using echoes, we assume a completely autonomous setup with (near) collocated microphone and the acoustic source. We first introduce a simple, easy to analyze estimator, and prove that the sequence of room and tra...
We present a new technique for estimating the specular peak of the bidirectional reflectance distribution function (BRDF) based on finite rate of innovation (FRI) sampling. The specular component of the BRDF varies rapidly, so it is challenging to acquire it by pointwise sampling. Yet, the knowledge of its precise location is key to render realisti...
Recent works on reconstruction of room geometry from echoes assume that the geometry of the sensor array is known. In this paper, we show that such an assumption is not essential; echoes provide sufficient clues to reconstruct the room’s and the array’s geometries jointly, even from a single acoustic event. Rather than focusing on the combinatorial...
We introduce in this paper the recursive Hessian sketch, a new adaptive filtering algorithm based on sketching the same exponentially weighted least squares problem solved by the recursive least squares algorithm. The algorithm maintains a number of sketches of the inverse autocorrelation matrix and recursively updates them at random intervals. The...
We propose a novel camera pose estimation or perspective-n-point (PnP) algorithm, based on the idea of consistency regions and half-space intersections. Our algorithm has linear time-complexity and a squared reconstruction error that decreases at least quadratically, as the number of feature point correspondences increase. Inspired by ideas from tr...
How can we decipher the hidden structure of a network based on limited observations? This question arises in many scenarios ranging from social to wireless and to neural networks. In such settings, we typically observe the nodes’ behaviors (e.g., the time a node learns about a piece of information, or the time a node gets infected by a disease), an...
Relightable photographs are alternatives to traditional photographs as they provide a richer viewing experience. However, the complex acquisition systems of existing techniques have restricted its usage to specialized setups. We introduce an easy-to-use and affordable solution for reflectance acquisition by using smartphones. Our goal is to enable...
We present a sampling theory for a class of binary images with finite rate of innovation (FRI). Every image in our model is the restriction of $\mathds{1}_{\{p\leq0\}}$ to the image plane, where $\mathds{1}$ denotes the indicator function and $p$ is some real bivariate polynomial. This particularly means that the boundaries in the image form a subs...
![Fig. 4. (a) Position of Orange's GSM antennas in Ivory Coast [35]. The...](profile/Jayakrishnan-Unnikrishnan/publication/283621097/figure/fig1/AS:614178679164945@1523443016268/a-Position-of-Oranges-GSM-antennas-in-Ivory-Coast-35-The-sub-prefectures-are-shown_Q320.jpg)
Most users of online services have unique behavioral or usage patterns. These
behavioral patterns can be exploited to identify and track users by using only
the observed patterns in the behavior. We study the task of identifying users
from statistics of their behavioral patterns. Specifically, we focus on the
setting in which we are given histogram...
Consider a set of probes, called “agents”, who sample, based on opportunistic contacts, a population moving between a set of discrete locations. An example of such agents are Bluetooth probes that sample the visible Bluetooth devices in a population. Based on the obtained measurements, we construct a parametric statistical model to jointly estimate...
We study the problem of solving a linear sensing system when the observations
are unlabeled. Specifically we seek a solution to a linear system of equations
y = Ax when the order of the observations in the vector y is unknown. Focusing
on the setting in which A is a random matrix with i.i.d. entries, we show that
if the sensing matrix A admits an o...
Chip designers place on-chip thermal sensors to measure local temperatures, thus preventing thermal runaway situations in many-core processing architectures. However, the quality of the thermal reconstruction is directly dependent on the number of placed sensors, which should be minimized, while guaranteeing full detection of all the worst case tem...
Euclidean distance matrices (EDMs) are matrices of the squared distances between points. The definition is deceivingly simple; thanks to their many useful properties, they have found applications in psychometrics, crystallography, machine learning, wireless sensor networks, acoustics, and more. Despite the usefulness of EDMs, they seem to be insuff...
We study the problem of solving a linear sensing system when the observations are unlabeled. Specifically we seek a solution to a linear system of equations y = Ax when the order of the observations in the vector y is unknown. Focusing on the setting in which A is a random matrix with i.i.d. entries, we show that if the sensing matrix A admits an o...
Although many advances have been made in light-field and camera-array image processing, there is still a lack of thorough analysis of the localisation accuracy of different multi-camera systems. By considering the problem from a frame-quantisation perspective, we are able to quantify the point localisation error of circular camera configurations. S...
Continuous-domain visual signals are usually captured as discrete (digital)
images. This operation is not invertible in general, in the sense that the
continuous-domain signal cannot be exactly reconstructed based on the discrete
image, unless it satisfies certain constraints (\emph{e.g.}, bandlimitedness).
In this paper, we study the problem of re...
Recent methods for localization of microphones in a microphone array exploit sound sources at a priori unknown locations. This is convenient for ad-hoc arrays, as it requires little additional infrastructure. We propose a flexible localization algorithm by first recognizing the problem as an instance of multidimensional unfolding (MDU)—a classical...
Energy efficiency of wireless sensor networks (WSNs) can be improved by moving base stations (BSs), as this scheme evenly distributes the communication load in the network. However, physically moving the BSs is complicated and costly. In this paper, we propose a new scheme: virtually moving the BSs. We deploy an excessive number of BSs and adaptive...
The geometry of room acoustics is such that the reverberant signal can be seen as the same waveform emitted from multiple locations. In analogy with the rake receiver from wireless com munications, we propose several beamforming strategies that exploit, rather than suppress, this additional spatio-temporal di versity. Unlike earlier work in the f...
Solving a linear inverse problem may include difficulties such as the presence of outliers and a mixing matrix with a large condition number. In such cases a regularized robust estimator is needed. We propose a new tau-type regularized robust estimator that is simultaneously highly robust against outliers, highly efficient in the presence of purely...
In architecture, space is traditionally understood as a cumulus of forms, colors and textures that form a conceived area, or—in other words—architectural space design is mainly achieved by following visual concepts. The human experience of space, however, is multi-modal where light and sound, i.e., eye and ear are the main transducers through which...
Euclidean distance matrices (EDM) are matrices of squared distances between
points. The definition is deceivingly simple: thanks to their many useful
properties they have found applications in psychometrics, crystallography,
machine learning, wireless sensor networks, acoustics, and more. Despite the
usefulness of EDMs, they seem to be insufficient...
We study a modification of the orthogonal matching pursuit (OMP) for estimating sparse multipath channels. The reflectors that generate the multipath components are not ideal; rather, they act as filters, so that the returned pulses are reshaped and widened. To deal with this, we introduce unknown filters into the OMP, and then search for the best...
Stained glass windows are designed to reveal their powerful artistry under diverse and time-varying lighting conditions; virtual relighting of stained glass, therefore represents an exceptional tool for the appreciation of this age old art form. However, as opposed to most other artifacts, stained glass windows are extremely difficult if not imposs...