Peter Balazs

Austrian Academy of Sciences (OeAW) | ÖAW · Acoustics Research Institute

Waveform-based deep learning faces a dilemma between nonparametric and parametric approaches. On one hand, convolutional neural networks (convnets) may approximate any linear time-invariant system; yet, in practice, their frequency responses become more irregular as their receptive fields grow. On the other hand, a parametric model such as LEAF is...

The paper uses a frame-theoretic setting to study the injectivity of a ReLU-layer on the closed ball of $\mathbb{R}^n$ and its non-negative part. In particular, the interplay between the radius of the ball and the bias vector is emphasized. Together with a perspective from convex geometry, this leads to a computationally feasible method of verifyin...

The recovery of a signal from the magnitudes of its transformation, like the Fourier transform, is known as the phase retrieval problem and is of big relevance in various fields of engineering and applied physics. In this paper, we present a fast inertial/momentum based algorithm for the phase retrieval problem and we prove a convergence guarantee...

The phase retrieval problem is found in various areas of applications of engineering and applied physics. It is also a very active field of research in mathematics, signal processing and machine learning. In this paper, we present an accelerated version of the well known Fast Griffin-Lim algorithm (FGLA) for the phase retrieval problem in a general...

Frames have been investigated frequently over the last few decades due to their valuable properties, which are desirable for various applications as well as interesting for theory. Some applications additionally require distributed processing techniques, which naturally leads to the concept of fusion frames and fusion frame systems. The latter cons...

Fusion frames are a very active area of research today because of their myriad of applications in pure mathematics, applied mathematics, engineering, medicine, signal and image processing and much more. They provide a great flexibility for designing sets of vectors for applications and are therefore prominent in all these areas, including e.g. miti...

Frames have been investigated frequently over the last few decades due to their valuable properties, which are desirable for various applications as well as interesting for theory. Some applications additionally require distributed processing techniques, which naturally leads to the concept of fusion frames and fusion frame systems. The latter cons...

We formulate a quantitative finite-dimensional conjecture about frame multipliers and prove that it is equivalent to Conjecture 1 in [SB2]. We then present solutions to the conjecture for certain classes of frame multipliers. In particular, we prove that there is a universal constant $\kappa>0$ so that for all $C,\beta>0$ and $N\in\mathbb{N}$ the f...

We introduce the affine ensemble, a class of determinantal point processes (DPP) in the half-plane C^+ associated with the ax+b (affine) group, depending on an admissible Hardy function {\psi}. We obtain the asymptotic behavior of the variance, the exact value of the asymptotic constant, and non-asymptotic upper and lower bounds for the variance on...

We provide the foundations of a Hilbert space theory for the short-time Fourier transform (STFT) where the flat tori T 2 N = R 2 /(Z × N Z) = [0, 1] × [0, N ] act as phase spaces. We work on an N-dimensional subspace S N of distributions periodic in time and frequency in the dual S 0 (R) of the Feichtinger algebra S 0 (R) and equip it with an inner...

We provide the foundations of a Hilbert space theory for the short-time Fourier transform (STFT) where the flat tori T 2 N = R 2 /(Z × N Z) = [0, 1] × [0, N ] act as phase spaces. We work on an N-dimensional subspace S N of distributions periodic in time and frequency in the dual S 0 (R) of the Feichtinger algebra S 0 (R) and equip it with an inner...

House mice communicate through ultrasonic vocalizations (USVs), which are above the range of human hearing (>20 kHz), and several automated methods have been developed for USV detection and classification. Here we evaluate their advantages and disadvantages in a full, systematic comparison, while also presenting a new approach. This study aims to 1...

Frames for Hilbert spaces are interesting for mathematicians but also important for applications in, e.g., signal analysis and physics. In both mathematics and physics, it is natural to consider a full scale of spaces, and not only a single one. In this paper, we study how certain frame-related properties of a certain sequence in one of the spaces,...

Seismologists have to deal with overlapping and noisy signals. Techniques such as source separation can be used to solve this problem. Over the past few decades, signal processing techniques used for source separation have advanced significantly for multi‐station settings. But not so many options are available when it comes to single‐station data....

We study the connection between STFT multipliers $A^{g_1,g_2}_{1\otimes m}$ having windows $g_1,g_2$, symbols $a(x,\omega)=(1\otimes m)(x,\omega)=m(\omega)$, $(x,\omega)\in\mathbb{R}^{2d}$, and the Fourier multipliers $T_{m_2}$ with symbol $m_2$ on $\mathbb{R}^d$. We find sufficient and necessary conditions on symbols $m,m_2$ and windows $g_1,g_2$...

Frames for Hilbert spaces are interesting for mathematicians but also important for applications e.g. in signal analysis and in physics. Both in mathematics and physics it is natural to consider a full scale of spaces, and not only a single one. In this paper, we study how certain frame-related properties, as completeness or the property of being a...

Continuous frames and tensor products are important topics in theoretical physics. This paper combines those concepts. We derive fundamental properties of continuous frames for tensor product of Hilbert spaces. This includes, for example, the consistency property, i.e. preservation of the frame property under the tensor product, and the description...

Finding the best K-sparse approximation of a signal in a redundant dictionary is an NP-hard problem. Suboptimal greedy matching pursuit (MP) algorithms are generally used for this task. In this work, we present an acceleration technique and an implementation of the matching pursuit algorithm acting on a multi-Gabor dictionary, i.e., a concatenation...

The scattering transform is a non-linear signal representation method based on cascaded wavelet transform magnitudes. In this paper we introduce phase scattering, a novel approach where we use phase derivatives in a scattering procedure. We first revisit phase-related concepts for representing time-frequency information of audio signals, in particu...

Audio inpainting refers to signal processing techniques that aim at restoring missing or corrupted consecutive samples in audio signals. Prior works have shown that $\ell_1$- minimization with appropriate weighting is capable of solving audio inpainting problems, both for the analysis and the synthesis models. These models assume that audio signals...

We introduce the affine ensemble, a class of determinantal point processes (DPP) in the half-plane C+ associated with the ax + b group, depending on an admissible Hardy function. We obtain the asymptotic behavior of the variance, the exact value of the asymptotic constant, and non-asymptotic upper and lower bounds for the variance on a compact set...

Seismologists have to deal with overlapping and noisy signals. Techniques such as source separation can be used to solve this problem. Over the past few decades, signal processing techniques used for source separation have advanced significantly for multi-station settings. But not so many options are available when it comes to single-station data....

We derive fundamental properties of continuous frames for tensor product of Hilbert spaces. This includes, for example, the consistency property, i.e. preservation of the frame property under the tensor product, and the description of canonical dual frames as inverses of the frame operator in the tensor product setting. We show the full characteriz...

Finding the best K -sparse approximation of a signal in a redundant dictionary is an NP-hard problem. Suboptimal greedy matching pursuit algorithms are generally used for this task. In this work, we present an acceleration technique and an implementation of the matching pursuit algorithm acting on a multi-Gabor dictionary, i.e., a concatenation of...

House mice communicate through ultrasonic vocalizations (USVs), which are above the range of human hearing (>20 kHz), and several automated methods have been developed for USV detection and classification. Here we evaluate their advantages and disadvantages in a full, systematic comparison. We compared the performance of four detection methods, Dee...

Zusammenfassung Für viele Anwendungen in der Akustik ist es notwendig, Signale und Funktionen mithilfe von zeitvarianten Filtern zu bearbeiten, z. B. um Komponenten aus einem Signal zu entfernen, deren Frequenzverlauf sich über die Zeit ändert. Es wird eine Methode vorgestellt, die auf einer Darstellung des Signals durch Rahmen (engl. Frames) basie...

The objective of audio inpainting is to fill a gap in an audio signal. This is ideally done by reconstructing the original signal or, at least, by inferring a meaningful surrogate signal. We propose a novel approach applying sparse modeling in the time-frequency (TF) domain. In particular, we devise a dictionary learning technique which learns the...

Males in a wide variety of taxa, including insects, birds and mammals, produce vocalizations to attract females. Male house mice emit ultrasonic vocalizations (USVs), especially during courtship and mating, which are surprising complex. It is often suggested that male mice vocalize at higher rates after interacting with a female, but the evidence i...

Matching pursuit (MP) algorithms are widely used greedy methods to find K-sparse signal approximations in redundant dictionaries. We present an acceleration technique and an implementation of the matching pursuit algorithm acting on a multi-Gabor dictionary , i.e., a concatenation of several Gabor-type time-frequency dictionaries, consisting of tra...

For applications like the numerical solution of physical equations a discretization scheme for operators is necessary. Recently frames have been used for such an operator representation. In this paper, we interpret the operator representation using fusion frames as a generalization of fusion Gram matrices. We present the basic definition of U-fusio...

For finding the numerical solution of operator equations in many applications a decomposition in subspaces is needed. Therefore, it is necessary to extend the known method of matrix representation to the utilization of fusion frames. In this paper we investigate this representation of operators on a Hilbert space $\Hil$ with Bessel fusion sequences...

The paper presents a survey over frame multipliers and related concepts. In particular, it includes a short motivation of why multipliers are of interest to consider, a review as well as extension of recent results, devoted to the unconditional convergence of multipliers, sufficient and/or necessary conditions for the invertibility of multipliers,...

We prove general kernel theorems for operators acting between coorbit spaces. These are Banach spaces associated to an integrable representation of a locally compact group and contain most of the usual function spaces (Besov spaces, modulation spaces, etc.). A kernel theorem describes the form of every bounded operator between a coorbit space of te...

Many species in diverse taxonomic groups, including rodents, bats, and insects, communicate with complex ultrasonic vocalizations (USVs) (>20 kHz). Two main components of processing and analyzing USV recordings include detection and classification of syllable types. Recently we developed an efficient algorithm for detecting mouse USVs (Automatic Mo...

In this contribution, we present a heuristic method to invert the reassigned short time Fourier transform magnitude spectrum to allow the reconstruction of the original time domain signal. This is a simple method just involving an additional smearing step before phase retrieval. Finally, we provide some numerical evidence that our method combined w...

We propose harmonic-aligned frame mask for speech signals using non-stationary Gabor transform (NSGT). A frame mask operates on the transfer coefficients of a signal and consequently converts the signal into a counterpart signal. It depicts the difference between the two signals. In preceding studies, frame masks based on regular Gabor transform we...

The Gram matrix is defined for Bessel sequences by combining synthesis with subsequent analysis operators. If different sequences are used and an operator U is inserted we reach so called U-cross Gram matrices. This can be seen as reinterpretation of the matrix representation of operators using frames. In this paper we investigate some necessary or...

In this paper, we analyze by means of simulations the applicability of random Gabor multipliers as compressive measurements. In particular, we consider signals that are sparse with respect to Fourier or Gabor dictionaries, i.e., signals that are sparse in frequency or time-frequency domains. This work is an extension of our earlier contribution, wh...

We prove general kernel theorems for operators acting between coorbit spaces. These are Banach spaces associated to an integrable representation of a locally compact group and contain most of the usual function spaces (Besov spaces, modulation spaces, etc.). A kernel theorem describes the form of every bounded operator between a coorbit space of te...

For the solution of operator equations, Stevenson introduced a definition of frames, where a Hilbert space and its dual are not identified. This means that the Riesz isomorphism is not used as an identification, which, for example, does not make sense for the Sobolev spaces and . In this article, we are going to revisit the concept of Stevenson fra...

For the solution of operator equations, Stevenson introduced a definition of frames, where a Hilbert space and its dual are {\em not} identified. This means that the Riesz isomorphism is not used as an identification, which, for example, does not make sense for the Sobolev spaces $H_0^1(\Omega)$ and $H^{-1}(\Omega)$. In this article, we are going t...

The Gram matrix can be defined for Bessel sequences by combining synthesis with subsequent analysis. If different sequences are used and an operator is inserted we reach so called U-cross Gram matrices. This can be seen as reinterpretation of the matrix representation of operators using frames. In this paper we investigate some necessary or suffcie...

The paper presents a survey over frame multipliers and related concepts. In particular, it includes a short motivation of why multipliers are of interest to consider, a review as well as extension of recent results, devoted to the unconditional convergence of multipliers, sufficient and/or necessary conditions for the invertibility of multipliers,...

We present a novel family of continuous, linear time-frequency transforms adaptable to a multitude of (nonlinear) frequency scales. Similar to classical time-frequency or time-scale representations, the representation coefficients are obtained as inner products with the elements of a continuously indexed family of time-frequency atoms. These atoms...

We present a novel method for the compensation of long duration data loss in audio signals, in particular music. The concealment of such signal defects is based on a graph that encodes signal structure in terms of time-persistent spectral similarity. A suitable candidate segment for the substitution of the lost content is proposed by an intuitive o...

Many audio applications rely on filter banks (FBs) to analyze, process, and re-synthesize sounds. For these applications, an important property of the analysis–synthesis system is the reconstruction error; it has to be minimized to avoid audible artifacts. Other advantageous properties include stability and low redundancy. To exploit some aspects o...

House mice (Mus musculus) emit ultrasonic vocalizations (USVs), which are surprisingly complex and have features of bird song, but their functions are not well understood. Previous studies have reported mixed evidence on whether there are sex differences in USV emission, though vocalization rate or other features may depend upon whether potential r...

Data: Total number of elements, frequencies and amplitudes of emitted USVs. (XLSX)

House mice (Mus musculus) emit complex ultrasonic vocalizations (USVs) during social and sexual interactions, which have features similar to bird song (i.e., they are composed of several different types of syllables, uttered in succession over time to form a pattern of sequences). Manually processing complex vocalization data is time-consuming and...

This review chapter aims to strengthen the link between frame theory and signal processing tasks in psychoacoustics. On the one side, the basic concepts of frame theory are presented and some proofs are provided to explain those concepts in some detail. The goal is to reveal to hearing scientists how this mathematical theory could be relevant for t...

This chapter offers a detailed survey on intrinsically localized frames and the corresponding matrix representation of operators. We re-investigate the properties of localized frames and the associated Banach spaces in full detail. We investigate the representation of operators using localized frames in a Galerkin-type scheme. We show how the bound...

A noniterative method for the reconstruction of the short-time fourier transform (STFT) phase from the magnitude is presented. The method is based on the direct relationship between the partial derivatives of the phase and the logarithm of the magnitude of the un-sampled STFT with respect to the Gaussian window. Although the theory holds in the con...

This paper is devoted to three aspects of the relation between reproducing kernel Hilbert spaces (RKHS) and stable analysis/synthesis processes: First, we analyze the structure of the reproducing kernel of a RKHS using frames and reproducing pairs. Second, we give a new proof for the result that finite redundancy of a continuous frame implies atomi...

During the process of writing the manuscript ["Continuous warped time-frequency representations - Coorbit spaces and discretization", N. Holighaus, C. Wiesmeyr and P. Balazs], the work ["Continuous Frames, Function Spaces and the Discretization Problem" by M. Fornasier and H. Rauhut - (1)] was one of the major foundations of our results and, natura...

We use the concept of reproducing pairs to study Gabor systems at critical density. First, we present a generalization of the Balian-Low theorem to the reproducing pairs setting. Then, we prove our main result that there exist a reproducing partner for the Gabor system of integer time-frequency shifts of the Gaussian. In other words, the coefficien...

To be able to solve operator equations numerically a discretization of those operators is necessary. In the Galerkin approach bases are used to achieve discretized versions of operators. In a more general set-up, frames can be used to sample the involved signal spaces and therefore those operators. Here we look at the redundant representation of op...

This chapter offers a detailed survey on intrinsically localized frames and the corresponding matrix representation of operators. We re-investigate the properties of localized frames and the associated Banach spaces in full detail. We investigate the representation of operators using localized frames in a Galerkin-type scheme. We show how the bound...

Many audio applications perform perception-based time-frequency (TF) analysis by decomposing sounds into a set of functions with good TF localization (i.e. with a small essential support in the TF domain) using TF transforms and applying psychoacoustic models of auditory masking to the transform coefficients. To accurately predict masking interacti...

Time-frequency masking function for a Gaussian TF atom with a center frequency of 4 kHz and a sensation level of 60 dB. The archive, compressed in a ZIP file (2.7 kB in size), includes the raw data in a MAT file, a Matlab/Octave function to interpolate the TF masking function at any sampling rate, and a “readme” text file. (ZIP)

This review chapter aims to strengthen the link between frame theory and signal processing tasks in psychoacoustics. On the one side, the basic concepts of frame theory are presented and some proofs are provided to explain those concepts in some detail. The goal is to reveal to hearing scientists how this mathematical theory could be relevant for t...

In this contribution, we present a method to compensate for long duration data gaps in audio signals, in particular music. To achieve this task, a similarity graph is constructed, based on a short-time Fourier analysis of reliable signal segments, e.g. the uncorrupted remainder of the music piece, and the temporal regions adjacent to the unreliable...

The α-modulation transform is a time-frequency transform generated by square-integrable representations of the affine Weyl-Heisenberg group modulo suitable subgroups. In this paper we prove new conditions that guarantee the admissibility of a given window function. We also show that the generalized coorbit theory can be applied to this setting, ass...

Many audio applications rely on filter banks (FBs) to analyze, process, and re-synthesize sounds. To approximate the auditory frequency resolution in the signal chain, some applications rely on perceptually motivated FBs, the gammatone FB being a popular example. However, most perceptually motivated FBs only allow partial signal reconstruction at h...

Recently it has been established that given an invertible frame multiplier with semi-normalized symbol, a specific dual of any of the two involved frames can be determined by the inversion process. The inverse can be represented as a multiplier with the reciprocal symbol, this particular dual of one of the given frames, and any dual of the other fr...

We consider the quantum dynamics of a charged particle evolving under the action of a constant homogeneous magnetic field, with emphasis on the discrete subgroups of the Heisenberg group (in the Euclidean case) and of the SL(2,IR) group (in the Hyperbolic case). We prove completeness properties for discrete coherent states associated with higher or...

The short-time Fourier transform (STFT) is a time-frequency representation widely used in applications, for example in audio signal processing. Recently it has been shown that not only the amplitude, but also the phase of this representation can be successfully exploited for improved analysis and processing. In this paper we describe a rather pecul...

In this paper we introduce and investigate the concept of repro- ducing pairs as a generalization of continuous frames. Reproducing pairs yield a bounded analysis and synthesis process while the frame condition can be omitted at both stages. Moreover, we will investigate certain continuous frames (resp. reproducing pairs) on LCA groups, which can b...

We describe ERB-MDCT, an invertible real-valued time-frequency transform based on MDCT, which is widely used in audio coding (e.g. MP3 and AAC). ERB-MDCT was designed similarly to ERBLet, a recent invertible transform with a resolution evolving across frequency to match the perceptual ERB frequency scale, while the frequency scale in most invertibl...