Independent Component Analysis by Entropy Bound Minimization

Dept. of CSEE, UMBC, Baltimore, MD, USA
IEEE Transactions on Signal Processing (Impact Factor: 2.79). 11/2010; 58(10):5151 - 5164. DOI: 10.1109/TSP.2010.2055859
Source: IEEE Xplore


A novel (differential) entropy estimator is introduced where the maximum entropy bound is used to approximate the entropy given the observations, and is computed using a numerical procedure thus resulting in accurate estimates for the entropy. We show that such an estimator exists for a wide class of measuring functions, and provide a number of design examples to demonstrate its flexible nature. We then derive a novel independent component analysis (ICA) algorithm that uses the entropy estimate thus obtained, ICA by entropy bound minimization (ICA-EBM). The algorithm adopts a line search procedure, and initially uses updates that constrain the demixing matrix to be orthogonal for robust performance. We demonstrate the superior performance of ICA-EBM and its ability to match sources that come from a wide range of distributions using simulated and real-world data.

Full-text preview

Available from:
    • "where 0.5 log(2πe) is the maximum entropy of a signal of unit variance (i.e. a gaussian signal) [10]. ERBM (Entropy Rate Bound Minimization) is an extension of EBM by Li and Adali [11]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Blind Source Separation (BSS) finds applications in audio scene analysis to identify and separate sources from instantaneous or convolutive mixtures. Independent Component Analysis (ICA) is notably a powerful tool for BSS allowing to decompose a multichannel recording into a set of independent components. However, real acoustic recordings also contain sound reflections which can be considered as secondary sources highly correlated to the direct sounds. In this paper, we propose to study the ability of the Entropy Rate Bound Minimization (ERBM) algorithm to separate direct sounds from reflections. The evaluation is conducted through a simplified audioconference scenario simulating a 2nd-order ambisonic microphone sound capture. Results are compared to classical ICA algorithms, including tensorial methods or entropy minimization algorithms. Objective measures show that, thanks to some assumptions on the sources’ model, the ERBM algorithm outperforms state-of-the-art methods, hence showing ability to separate cross-correlated speech sources and then direct and reflected signals.
    No preview · Conference Paper · Feb 2016
  • Source
    • "The estimated bound of entropy is minimized to find the ICAs. The algorithm adopts a line search procedure initially constraining the demixing matrix to be orthogonal (Li and Adali, 2010b). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Independent component analysis (ICA) techniques offer a data-driven possibility to analyze brain functional MRI data in real-time. Typical ICA methods used in functional magnetic resonance imaging (fMRI), however, have been until now mostly developed and optimized for the off-line case in which all data is available. Real-time experiments are ill-posed for ICA in that several constraints are added: limited data, limited analysis time and dynamic changes in the data and computational speed. Previous studies have shown that particular choices of ICA parameters can be used to monitor real-time fMRI (rt-fMRI) brain activation, but it is unknown how other choices would perform. In this rt-fMRI simulation study we investigate and compare the performance of 14 different publicly available ICA algorithms systematically sampling different growing window lengths (WLs), model order (MO) as well as a priori conditions (none, spatial or temporal). Performance is evaluated by computing the spatial and temporal correlation to a target component as well as computation time. Four algorithms are identified as best performing (constrained ICA, fastICA, amuse, and evd), with their corresponding parameter choices. Both spatial and temporal priors are found to provide equal or improved performances in similarity to the target compared with their off-line counterpart, with greatly reduced computation costs. This study suggests parameter choices that can be further investigated in a sliding-window approach for a rt-fMRI experiment.
    Full-text · Article · Feb 2013 · Frontiers in Human Neuroscience
  • Source
    • "led to ICA algorithms such as EFICA [23] and adaptive complex maximization of non-Gaussianity [26]. ICA by entropy bound minimization (ICA-EBM) [21] use an efficient entropy estimator based on the bounding of the entropy estimates, and with only a few measuring functions, can approximate the pdf of a wide range of densities including sub-or super-Gaussian, unimodal or multimodal, symmetric or skewed distributions. A more powerful class takes both sample correlation and non- Gaussianity into account. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Since the discovery of functional connectivity in fMRI data (i.e., temporal correlations between spatially distinct regions of the brain) there has been a considerable amount of work in this field. One important focus has been on the analysis of brain connectivity using the concept of networks instead of regions. Approximately ten years ago, two important research areas grew out of this concept. First, a network proposed to be "a default mode of brain function" since dubbed the default mode network was proposed by Raichle. Secondly, multisubject or group independent component analysis (ICA) provided a data-driven approach to study properties of brain networks, including the default mode network. In this paper, we provide a focused review of how ICA has contributed to the study of intrinsic networks. We discuss some methodological considerations for group ICA and highlight multiple analytic approaches for studying brain networks. We also show examples of some of the differences observed in the default mode and resting networks in the diseased brain. In summary, we are in exciting times and still just beginning to reap the benefits of the richness of functional brain networks as well as available analytic approaches.
    Full-text · Article · Dec 2012
Show more