Conference Paper

# Fast adaptive variational sparse Bayesian learning with automatic relevance determination.

DOI: 10.1109/ICASSP.2011.5946760 Conference: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2011, May 22-27, 2011, Prague Congress Center, Prague, Czech Republic

Source: DBLP

- [Show abstract] [Hide abstract]

**ABSTRACT:**In this paper we provide an algorithm allowing to solve the variational Bayesian issue as a functional optimization problem. The main contribution of this paper is to transpose a classical iterative algorithm of optimization in the metric space of probability densities involved in the Bayesian methodology. The main advantage of this methodology is that it allows to address large dimensional inverse problems by unsupervised algorithms. The interest of our algorithm is enhanced by its application to large dimensional linear inverse problems involving sparse objects. Finally, we provide simulation results. First we show the good numerical performances of our method by comparing it with classical ones on a small tomographic problem. On a second time we treat a large dimensional dictionary learning problem and compare our method with a wavelet based one. keywords: ill-posed inverse problems, variational bayesian methodol-ogy, sparse signal reconstruction, infinite dimensional convex optimization 1. Introduction. The recent development of information technologies has in-creased the expansion of inverse problems for very large dimensional datasets. Indeed whereas the 90's decade have seen the introduction of image reconstruction problems, the current main interest is on 3D sequences (3D + T), thus on large dimensional sets of data. There is therefore a significant growth in the number of measurements in the involved problems. One has frequently to treat the reconstruction of more than one million data. At the same time, the signal processing techniques have helped to over-come the limitations of measuring instruments as they supplied the design of systems involving indirect measures. These new equipments introduced in exchange novel sig-nal processing challenges, such as super resolution deconvolution, source separation or tomographic reconstruction. All these problems are ill posed, the only information contained in the data and in the model of acquisition are not sufficient to obtain a good estimation of the unknown objects.SIAM Journal on Imaging Sciences 10/2014; 7(4). · 2.87 Impact Factor - [Show abstract] [Hide abstract]

**ABSTRACT:**In this work a new online learning algorithm that uses automatic relevance determination (ARD) is proposed for fast adaptive non linear filtering. A sequential decision rule for inclusion or deletion of basis functions is obtained by applying a recently proposed fast variational sparse Bayesian learning (SBL) method. The proposed scheme uses a sliding window estimator to process the data in an online fashion. The noise variance can be implicitly estimated by the algorithm. It is shown that the described method has better mean square error (MSE) performance than a state of the art kernel re cursive least squares (Kernel-RLS) algorithm when using the same number of basis functions.Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on; 06/2011 - [Show abstract] [Hide abstract]

**ABSTRACT:**In this work, a variational Bayesian framework for efficient training of echo state networks (ESNs) with automatic regularization and delay&sum (D&S) readout adaptation is proposed. The algorithm uses a classical batch learning of ESNs. By treating the network echo states as fixed basis functions parameterized with delay parameters, we propose a variational Bayesian ESN training scheme. The variational approach allows for a seamless combination of sparse Bayesian learning ideas and a variational Bayesian space-alternating generalized expectation-maximization (VB-SAGE) algorithm for estimating parameters of superimposed signals. While the former method realizes automatic regularization of ESNs, which also determines which echo states and input signals are relevant for "explaining" the desired signal, the latter method provides a basis for joint estimation of D&S readout parameters. The proposed training algorithm can naturally be extended to ESNs with fixed filter neurons. It also generalizes the recently proposed expectation-maximization-based D&S readout adaptation method. The proposed algorithm was tested on synthetic data prediction tasks as well as on dynamic handwritten character recognition.Neural Computation 12/2011; 24(4):967-95. · 1.69 Impact Factor

Data provided are for informational purposes only. Although carefully collected, accuracy cannot be guaranteed. The impact factor represents a rough estimation of the journal's impact factor and does not reflect the actual current impact factor. Publisher conditions are provided by RoMEO. Differing provisions from the publisher's actual policy or licence agreement may be applicable.