-
[show abstract]
[hide abstract]
ABSTRACT: The plenoptic function (POF) provides a powerful conceptual tool for describing a number of problems in image/video processing, vision, and graphics. For example, image-based rendering is shown as sampling and interpolation of the POF. In such applications, it is important to characterize the bandwidth of the POF. We study a simple but representative model of the scene where band-limited signals (e.g., texture images) are "painted" on smooth surfaces (e.g., of objects or walls). We show that, in general, the POF is not band limited unless the surfaces are flat. We then derive simple rules to estimate the essential bandwidth of the POF for this model. Our analysis reveals that, in addition to the maximum and minimum depths and the maximum frequency of painted signals, the bandwidth of the POF also depends on the maximum surface slope. With a unifying formalism based on multidimensional signal processing, we can verify several key results in POF processing, such as induced filtering in space and depth-corrected interpolation, and quantify the necessary sampling rates.
IEEE Transactions on Image Processing 08/2011; 21(2):708-17. · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: This paper presents a signal processing tool for analyzing and manipulating digitized acoustic wave fields, based on a spatio-temporal extension of the time-frequency representation space. The emphasis is on wave fields acquired with a 1-D linear array of equidistant microphones (representing the spatial samples), but the basic formulation is valid for the three dimensions of space. We start by defining a spatio-temporal extension of the short time Fourier transform, in both continuous and discrete space and time, and then focus on applications of the proposed representation. In particular, we show that acoustic scenes with multiple sources can benefit from the use of space-frequency analysis in applications such as source separation (spatial filtering) and spatial audio coding (wave field coding). The experiments suggest that there is a spatial window size for which the performance of filtering and coding is optimal.
Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Consider a nonparametric representation of acoustic wave fields that consists of observing the sound pressure along a straight line or a smooth contour L defined in space. The observed data contains implicit information of the surrounding acoustic scene, both in terms of spatial arrangement of the sources and their respective temporal evolution. We show that such data can be effectively analyzed and processed in what we call the space-time-frequency representation space, consisting of a Gabor representation across the spatio-temporal manifold defined by the spatial axis L and the temporal axis t . In the presence of a source, the spectral patterns generated at L have a characteristic triangular shape that changes according to certain parameters, such as the source distance and direction, the number of sources, the concavity of L , and the analysis window size. Yet, in general, the wave fronts can be expressed as a function of elementary directional components-most notably, plane waves and far-field components. Furthermore, we address the problem of processing the wave field in discrete space and time, i.e., sampled along L and t , where a Gabor representation implies that the wave fronts are processed in a block-wise fashion. The key challenge is how to chose and customize a spatio-temporal filter bank such that it exploits the physical properties of the wave field while satisfying strict requirements such as perfect reconstruction, critical sampling, and computational efficiency. We discuss the architecture of such filter banks, and demonstrate their applicability in the context of real applications, such as spatial filtering, deconvolution, and wave field coding.
IEEE Transactions on Signal Processing 10/2010; · 2.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We revisit the topics of near-field adaptive beamforming and source localization following an alternative approach based on a spatio-temporal spectral representation of the acoustic wave field. With the proposed method, the wave field is expressed as a separable combination of the signal and spatial components that characterize the various sources in the acoustic scene. This allows beamforming operations such as beam steering and sidelobe canceling to be translated into a two-dimensional (2D) sampling problem, where the sampling kernels are derived according to a parametric model representing the 2D spectral pattern generated in the presence of a source. Conversely, the spectral pattern can be estimated from an arbitrary input through the use of parametric spectral estimation techniques, providing a novel solution to the near-field source localization problem.
Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on; 04/2010 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We address the problem of integrating directional analysis of sound into the filterbank of a spatial audio coder, with the purpose of processing and coding with some degree of independence the plane waves traveling in different directions. A plane wave represents an elementary waveform in the spatio-temporal analysis of the sound field, the same way a complex exponential is an elementary waveform in the time domain analysis of signals. Since a two-dimensional separable filterbank is not flexible enough for this purpose, we propose a non-separable approach based on the quincunx filterbank with diamond-shaped filters, cascaded with a base transform filterbank. This solution provides an invertible and critically sampled decomposition of the spatio-temporal spectra into subbands representing the different directions of wave propagation.
Applications of Signal Processing to Audio and Acoustics, 2009. WASPAA '09. IEEE Workshop on; 11/2009
-
[show abstract]
[hide abstract]
ABSTRACT: Classical digital signal processing (DSP) lore tells us the tale of a continuous-time primeval signal, of its brutal sampling, and of the magic sine interpolation that, under the aegis of bandlimitedness, brings the original signal back to (continuous) life. This article switches the conventional viewpoint and cast discrete-time sequences in the lead role, with continuous-time signals entering the scene as a derived version of their gap-toothed archetypes. Some well-known but seldom-taught facts about interpolation and vector spaces are brought together and the classic sine reconstruction formula derived naturally from the Lagrange interpolation method are recalled. Such an elegant and mathematically simple result can have a great educational value in building a solid yet very intuitive grasp of the interplay between analog and digital signals.
IEEE Signal Processing Magazine 10/2009; · 4.07 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: What should we do to raise the quality of signal processing publications to an even higher level? We believe it to be crucial to maintain the precision in describing our work in publications, ensured through a high-quality reviewing process. We also believe that if the experiments are performed on a large data set, the algorithm is compared to the state-of-the-art methods, the code and/or data are well documented and available online, we will all benefit and make it easier to build upon each other's work. It is a clear win-win situation for our community: we will have access to more and more algorithms and can spend time inventing new things rather than recreating existing ones.
IEEE Signal Processing Magazine 06/2009; · 4.07 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Hearing aids are electronic, battery-operated sensing devices which aim at compensating various kinds of hearing impairments. Recent advances in low-power electronics coupled with progresses made in digital signal processing offer the potential for substantial improvements over state-of-the-art systems. Nevertheless, efficient noise reduction in complex listening scenarios remains a challenging task, partly due to the limited number of microphones that can be integrated on such devices. We investigate the noise reduction capability of hearing instruments that may exchange data by means of a rate-constrained wireless link and thus benefit from the signals recorded at both ears of the user. We provide the necessary theoretical results to analyze this collaboration mechanism under two different coding strategies. The first approach takes full benefit of the binaural correlation, while the second neglects it, since binaural statistics are difficult to estimate in a practical setting. The gain achieved by collaborating hearing aids as a function of the communication bit rate is then characterized, both in a monaural and a binaural configuration. The corresponding optimal rate allocation strategies are computed in closed form. While the analytical derivation is limited to a simple acoustic scenario, the latter is shown to capture many of the features of the general problem. In particular, it is observed that the loss incurred by coding schemes which do not consider the binaural correlation is rather negligible in a very noisy environment. Finally, numerical results obtained using real measurements corroborate the potential of our approach in a realistic scenario.
IEEE Transactions on Signal Processing 03/2009; · 2.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Super-resolution algorithms combine multiple low resolution images into a single high resolution image. They have received a lot of attention recently in various application domains such as HDTV, satellite imaging, and video surveillance. These techniques take advantage of the aliasing present in the input images to reconstruct high frequency information of the resulting image. One of the major challenges in such algorithms is a good alignment of the input images: subpixel precision is required to enable accurate reconstruction. In this paper, we give an overview of some subspace techniques that address this problem. We first formulate super-resolution in a multichannel sampling framework with unknown offsets. Then, we present three registration methods: one approach using ideas from variable projections, one using a Fourier description of the aliased signals, and one using a spline description of the sampling kernel. The performance of the algorithms is evaluated in numerical simulations.
Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on; 11/2008
-
[show abstract]
[hide abstract]
ABSTRACT: Distributed estimation of an unknown signal is a common task in sensor networks. The scenario usually envisioned consists of several nodes, each making an observation correlated with the signal of interest. The acquired data is then wirelessly transmitted to a fusion center that aims at estimating the desired signal within a prescribed accuracy. Motivated by the obvious processing limitations inherent to such distributed infrastructures, we seek to find efficient compression schemes that account for limited available power and communication bandwidth. In this paper, we propose a transform-based approach to this problem where each sensor provides the fusion center with a low-dimensional approximation of its local observation by means of a suitable linear transform. Under the mean squared error criterion, we derive the optimal solution to apply at one sensor assuming all else being fixed. This naturally leads to an iterative algorithm whose optimality properties are exemplified using a simple though illustrative correlation model. The stationarity issue is also investigated. Under restrictive assumptions, we then provide an asymptotic distortion analysis, as the size of the observed vectors becomes large. Our derivation relies on a variation of the Toeplitz distribution theorem, which allows us to provide a reverse ldquowater-fillingrdquo perspective to the problem of optimal dimensionality reduction. We illustrate, with a first-order Gauss-Markov model, how our findings allow for the computation of analytical closed-form distortion formulas that provide an accurate estimation of the reconstruction error obtained in the finite-dimensional regime.
IEEE Transactions on Information Theory 05/2008; · 3.01 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Sparse sampling of continuous-time sparse signals is addressed. In particular, it is shown that sampling at the rate of innovation is possible, in some sense applying Occam's razor to the sampling of sparse signals. The noisy case is analyzed and solved, proposing methods reaching the optimal performance given by the Cramer-Rao bounds. Finally, a number of applications have been discussed where sparsity can be taken advantage of. The comprehensive coverage given in this article should lead to further research in sparse sampling, as well as new applications. One main application to use the theory presented in this article is ultra-wide band (UWB) communications.
IEEE Signal Processing Magazine 04/2008; · 4.07 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Directionlets are built as basis functions of critically sampled perfect-reconstruction transforms with directional vanishing moments imposed along different directions. We combine the directionlets with the space-frequency quantization (SFQ) image compression method, originally based on the standard two-dimensional wavelet transform. We show that our new compression method outperforms the standard SFQ as well as the state-of-the-art image compression methods, such as SPIHT and JPEG-2000, in terms of the quality of compressed images, especially in a low-rate compression regime. We also show that the order of computational complexity remains the same, as compared to the complexity of the standard SFQ algorithm.
Information, Communications & Signal Processing, 2007 6th International Conference on; 01/2008
-
IEEE Transactions on Information Theory 12/2007; 53(11):4400-4400. · 3.01 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In many applications, the sampling frequency is limited by the physical characteristics of the components: the pixel pitch, the rate of the analog-to-digital (AID) converter, etc. A low- pass filter is usually applied before the sampling operation to avoid aliasing. However, when multiple copies are available, it is possible to use the information that is inherently present in the aliasing to reconstruct a higher resolution signal. If the different copies have unknown relative offsets, this is a nonlinear problem in the offsets and the signal coefficients. They are not easily separable in the set of equations describing the super-resolution problem. Thus, we perform joint registration and reconstruction from multiple unregistered sets of samples. We give a mathematical formulation for the problem when there are M sets of N samples of a signal that is described by L expansion coefficients. We prove that the solution of the registration and reconstruction problem is generically unique if MN ges L + M - 1. We describe two subspace-based methods to compute this solution. Their complexity is analyzed, and some heuristic methods are proposed. Finally, some numerical simulation results on one- and two-dimensional signals are given to show the performance of these methods.
IEEE Transactions on Signal Processing 08/2007; · 2.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: How often have you been able to implement an algorithm as it is described in a paper? And when you did, were you confident that you had exactly the same parameter values and results as the authors of the paper? All too often, articles do not describe all the details of an algorithm and thus prohibit an implementation by someone else. In this paper, we describe our experience with reproducible research, a paradigm to allow other people to reproduce with minimal effort the results we have obtained. We discuss both the reproducibility of data and algorithms, and give examples for each of them. The effort required to make research reproducible is compensated by a higher visibility and impact of the results
Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on; 05/2007 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The Karhunen-Loeve transform (KLT) is a key element of many signal processing and communication tasks. Many recent applications involve distributed signal processing, where it is not generally possible to apply the KLT to the entire signal; rather, the KLT must be approximated in a distributed fashion. This paper investigates such distributed approaches to the KLT, where several distributed terminals observe disjoint subsets of a random vector. We introduce several versions of the distributed KLT. First, a local KLT is introduced, which is the optimal solution for a given terminal, assuming all else is fixed. This local KLT is different and in general improves upon the marginal KLT which simply ignores other terminals. Both optimal approximation and compression using this local KLT are derived. Two important special cases are studied in detail, namely, the partial observation KLT which has access to a subset of variables, but aims at reconstructing them all, and the conditional KLT which has access to side information at the decoder. We focus on the jointly Gaussian case, with known correlation structure, and on approximation and compression problems. Then, the distributed KLT is addressed by considering local KLTs in turn at the various terminals, leading to an iterative algorithm which is locally convergent, sometimes reaching a global optimum, depending on the overall correlation structure. For compression, it is shown that the classical distributed source coding techniques admit a natural transform coding interpretation, the transform being the distributed KLT. Examples throughout illustrate the performance of the proposed distributed KLT. This distributed transform has potential applications in sensor networks, distributed image databases, hyper-spectral imagery, and data fusion
IEEE Transactions on Information Theory 01/2007; · 3.01 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Consider the problem of sampling signals which are not bandlimited, but still have a finite number of degrees of freedom per unit of time, such as, for example, piecewise polynomial or piecewise sinusoidal signals, and call the number of degrees of freedom per unit of time the rate of innovation. Classical sampling theory does not enable a perfect reconstruction of such signals since they are not bandlimited. In this paper, we show that many signals with finite rate of innovation can be sampled and perfectly reconstructed using kernels of compact support and a local reconstruction algorithm. The class of kernels that we can use is very rich and includes functions satisfying strang-fix conditions, exponential splines and functions with rational Fourier transforms. Extension of such results to the 2-dimensional case are also discussed and an application to image super-resolution is presented
Image Processing, 2006 IEEE International Conference on; 11/2006
-
IEEE/ACM Transactions on Networking 11/2006; 14(5):1150- 1150. · 2.03 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Hearing aids are audio capture devices which aim at providing the hearing impaired with better audibility. Most of the state-of-the-art systems involve sensing devices that work independently. However, the availability of a wireless communication link between two hearing aids would allow to perform spatial beamforming such as to provide better rejection of interfering signals. In this paper, we identify and study the above scenario from an information-theoretic viewpoint. We explore the gain provided by collaborating hearing aids as a function of the communication rate. In particular, we derive a closed-form gain-rate formula in the case where a sound source has to be extracted from ambient noise. A similar analysis is provided in the presence of an interfering point source and the corresponding optimal rate allocation is discussed
Information Theory, 2006 IEEE International Symposium on; 08/2006
-
[show abstract]
[hide abstract]
ABSTRACT: To illustrate the conceptual issues related to sampling, source representation/coding and communication in sensor networks, we review the underlying theory and discuss specific examples. We show how the structure of the distributed sensing and communication problem dictates new processing architectures. The key challenge lies in the discretization of space, time and amplitude, since most of the advanced signal processing systems operate in discrete domain.
IEEE Signal Processing Magazine 08/2006; · 4.07 Impact Factor
