Independent Component Analysis by Entropy Bound Minimization

Dept. of CSEE, UMBC, Baltimore, MD, USA
IEEE Transactions on Signal Processing (Impact Factor: 2.81). 11/2010; DOI: 10.1109/TSP.2010.2055859
Source: IEEE Xplore

ABSTRACT A novel (differential) entropy estimator is introduced where the maximum entropy bound is used to approximate the entropy given the observations, and is computed using a numerical procedure thus resulting in accurate estimates for the entropy. We show that such an estimator exists for a wide class of measuring functions, and provide a number of design examples to demonstrate its flexible nature. We then derive a novel independent component analysis (ICA) algorithm that uses the entropy estimate thus obtained, ICA by entropy bound minimization (ICA-EBM). The algorithm adopts a line search procedure, and initially uses updates that constrain the demixing matrix to be orthogonal for robust performance. We demonstrate the superior performance of ICA-EBM and its ability to match sources that come from a wide range of distributions using simulated and real-world data.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Independent component analysis (ICA) techniques offer a data-driven possibility to analyze brain functional MRI data in real-time. Typical ICA methods used in functional magnetic resonance imaging (fMRI), however, have been until now mostly developed and optimized for the off-line case in which all data is available. Real-time experiments are ill-posed for ICA in that several constraints are added: limited data, limited analysis time and dynamic changes in the data and computational speed. Previous studies have shown that particular choices of ICA parameters can be used to monitor real-time fMRI (rt-fMRI) brain activation, but it is unknown how other choices would perform. In this rt-fMRI simulation study we investigate and compare the performance of 14 different publicly available ICA algorithms systematically sampling different growing window lengths (WLs), model order (MO) as well as a priori conditions (none, spatial or temporal). Performance is evaluated by computing the spatial and temporal correlation to a target component as well as computation time. Four algorithms are identified as best performing (constrained ICA, fastICA, amuse, and evd), with their corresponding parameter choices. Both spatial and temporal priors are found to provide equal or improved performances in similarity to the target compared with their off-line counterpart, with greatly reduced computation costs. This study suggests parameter choices that can be further investigated in a sliding-window approach for a rt-fMRI experiment.
    Frontiers in Human Neuroscience 01/2013; 7:19. · 2.91 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Signals of interest (SOIs) extraction are a vital issue in the field of communication signal processing. A promising approach is constrained independent component analysis (cICA). This paper extends the conventional constrained independent component analysis framework to the case of complex-valued mixing model and presents different prior information and different ways to be incorporated into the cICA framework. Two examples are demonstrated, ICA with cyclostationary constraint (ICA-CC) and ICA with spatial constraint (ICA-SC). The adaptive solution using the gradient ascent learning process is derived to solve the new constrained optimization problem in the ICA-CC example, while the rough spatial information corresponding to the direction of arrival (DOA) of the SOI can be utilized to select the specific initial vector for the desired solution before the learning process in the ICA-SC example. The corresponding experiment results show the efficacy and accuracy of the proposed algorithms.
    Neurocomputing. 02/2013; 101:204–216.
  • [Show abstract] [Hide abstract]
    ABSTRACT: It is seemingly paradoxical to the classical definition of the independent component analysis (ICA), that in reality, the true sources are often not strictly uncorrelated. With this in mind, this letter concerns a framework to extract quasi-uncorrelated sources with finite supports by optimizing a range-based contrast function under unit-norm constraints (to handle the inherent scaling indeterminacy of ICA) but without orthogonality constraints. Albeit the appealing contrast properties of the range-based function (e.g., the absence of mixing local optima), the function is not differentiable everywhere. Unfortunately, there is a dearth of literature on derivative-free optimizers that effectively handle such a nonsmooth yet promising contrast function. This is the compelling reason for the design of a nonsmooth optimization algorithm on a manifold of matrices having unit-norm columns with the following objectives: to ascertain convergence to a Clarke stationary point of the contrast function and adhere to the necessary unit-norm constraints more naturally. The proposed nonsmooth optimization algorithm crucially relies on the design and analysis of an extension of the mesh adaptive direct search (MADS) method to handle locally Lipschitz objective functions defined on the sphere. The applicability of the algorithm in the ICA domain is demonstrated with simulations involving natural, face, aerial, and texture images.
    Neural Computation 06/2013; · 1.76 Impact Factor


Available from

Tülay Adali