-
[show abstract]
[hide abstract]
ABSTRACT: In this paper we introduce a kernel-based recursive least-squares (KRLS) algorithm that is able to track nonlinear, time-varying relationships in data. To this purpose we first derive the standard KRLS equations from a Bayesian perspective (including a principled approach to pruning) and then take advantage of this framework to incorporate forgetting in a consistent way, thus enabling the algorithm to perform tracking in non-stationary scenarios. In addition to this tracking ability, the resulting algorithm has a number of appealing properties: It is online, requires a fixed amount of memory and computation per time step and incorporates regularization in a natural manner. We include experimental results that support the theory as well as illustrate the efficiency of the proposed algorithm.
Machine Learning for Signal Processing (MLSP), 2011 IEEE International Workshop on; 10/2011
-
[show abstract]
[hide abstract]
ABSTRACT: This paper introduces a new support vector machine (SVM) formulation to obtain sparse solutions in the primal SVM parameters, providing a new method for feature selection based on SVMs. This new approach includes additional constraints to the classical ones that drop the weights associated to those features that are likely to be irrelevant. A ν-SVM formulation has been used, where ν indicates the fraction of features to be considered. This paper presents two versions of the proposed sparse classifier, a 2-norm SVM and a 1-norm SVM, the latter having a reduced computational burden with respect to the first one. Additionally, an explanation is provided about how the presented approach can be readily extended to multiclass classification or to problems where groups of features, rather than isolated features, need to be selected. The algorithms have been tested in a variety of synthetic and real data sets and they have been compared against other state of the art SVM-based linear feature selection methods, such as 1-norm SVM and doubly regularized SVM. The results show the good feature selection ability of the approaches.
IEEE Transactions on Neural Networks 09/2011; · 2.95 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: In this correspondence, we derive an online adaptive one-class support vector machine. The machine structure is updated via growing and pruning mechanisms and the weights are updated using structural risk minimization principles underlying support vector machines. Our approach leads to very compact machines compared to other online kernel methods whose size, unless truncated, grows almost linearly with the number of observed patterns. The proposed method is online in the sense that every pattern is only presented once to the machine and there is no need to store past samples and adaptive in the sense that it can forget past input patterns and adapt to the new characteristics of the incoming data. Thus, the characterizing properties of our algorithm are compactness, adaptiveness and real-time processing capabilities, making it especially well-suited to solve online novelty detection problems. Regarding algorithm performance, we have carried out experiments in a time series segmentation problem, obtaining favorable results in both accuracy and model complexity with respect to two existing state-of-the-art methods.
IEEE Transactions on Signal Processing 07/2011; · 2.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: For regression tasks, traditional neural networks (NNs) have been superseded by Gaussian processes, which provide probabilistic predictions (input-dependent error bars), improved accuracy, and virtually no overfitting. Due to their high computational cost, in scenarios with massive data sets, one has to resort to sparse Gaussian processes, which strive to achieve similar performance with much smaller computational effort. In this context, we introduce a mixture of NNs with marginalized output weights that can both provide probabilistic predictions and improve on the performance of sparse Gaussian processes, at the same computational cost. The effectiveness of this approach is shown experimentally on some representative large data sets.
IEEE Transactions on Neural Networks 09/2010; · 2.95 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: It is a well-known result of estimation theory that biased estimators can outperform unbiased ones in terms of expected quadratic error. In steady state, many adaptive filtering algorithms offer an unbiased estimation of both the reference signal and the unknown true parameter vector. In this correspondence, we propose a simple yet effective scheme for adaptively biasing the weights of adaptive filters using an output multiplicative factor. We give theoretical results that show that the proposed configuration is able to provide a convenient bias versus variance tradeoff, leading to reductions in the filter mean-square error, especially in situations with a low signal-to-noise ratio (SNR). After reinterpreting the biased estimator as the combination of the original filter and a filter with constant output equal to 0, we propose practical schemes to adaptively adjust the multiplicative factor. Experiments are carried out for the normalized least-mean-squares (NLMS) adaptive filter, improving its mean-square performance in stationary situations and during the convergence phase.
IEEE Transactions on Signal Processing 08/2010; · 2.63 Impact Factor
