Independent Component Analysis by Entropy Bound Minimization

Dept. of CSEE, UMBC, Baltimore, MD, USA
IEEE Transactions on Signal Processing (Impact Factor: 2.81). 11/2010; DOI: 10.1109/TSP.2010.2055859
Source: IEEE Xplore

ABSTRACT A novel (differential) entropy estimator is introduced where the maximum entropy bound is used to approximate the entropy given the observations, and is computed using a numerical procedure thus resulting in accurate estimates for the entropy. We show that such an estimator exists for a wide class of measuring functions, and provide a number of design examples to demonstrate its flexible nature. We then derive a novel independent component analysis (ICA) algorithm that uses the entropy estimate thus obtained, ICA by entropy bound minimization (ICA-EBM). The algorithm adopts a line search procedure, and initially uses updates that constrain the demixing matrix to be orthogonal for robust performance. We demonstrate the superior performance of ICA-EBM and its ability to match sources that come from a wide range of distributions using simulated and real-world data.

  • [Show abstract] [Hide abstract]
    ABSTRACT: Signals of interest (SOIs) extraction are a vital issue in the field of communication signal processing. A promising approach is constrained independent component analysis (cICA). This paper extends the conventional constrained independent component analysis framework to the case of complex-valued mixing model and presents different prior information and different ways to be incorporated into the cICA framework. Two examples are demonstrated, ICA with cyclostationary constraint (ICA-CC) and ICA with spatial constraint (ICA-SC). The adaptive solution using the gradient ascent learning process is derived to solve the new constrained optimization problem in the ICA-CC example, while the rough spatial information corresponding to the direction of arrival (DOA) of the SOI can be utilized to select the specific initial vector for the desired solution before the learning process in the ICA-SC example. The corresponding experiment results show the efficacy and accuracy of the proposed algorithms.
    Neurocomputing. 02/2013; 101:204–216.
  • [Show abstract] [Hide abstract]
    ABSTRACT: I briefly outline our current understanding of the neutrino flavor conversions inside a supernova, describe the possible signals the neutrino detectors should look for, and comment on the extent to which these signals would reveal the neutrino mixing pattern, primary fluxes, and supernova dynamics.
    Nuclear Physics B - Proceedings Supplements 01/2011; 217(1):146-148. · 0.88 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: When prior knowledge is available, it is beneficial to use constrained blind source separation (BSS) algorithms that can utilize more information to distinguish the desired source from artifacts and noise. This paper proposes a one-unit second-order blind identification with reference (SOBI-R) algorithm for short transient signal extraction, which reformulates the conventional second-order blind identification (SOBI) algorithm in an iterative manner to achieve joint diagonalization and the reference information incorporated. The proposed algorithm was applied to single trial extraction of somatosensory evoked potential (SEP). The experimental results demonstrated its effectiveness. Compared with other algorithms including the autoregressive model with exogenous input (ARX), artificial neural networks (ANN) and one-unit the independent component analysis with reference (ICA-R), the proposed SOBI-R algorithm shows high robustness under conditions with low signal-to-noise ratios and less sensitivity to the reference signal.
    Information Sciences 12/2012; 227:90–101. · 3.64 Impact Factor


Available from

Tülay Adali