Thesis

Independent component analysis and beyond

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

'Independent component analysis' (ICA) ist ein Werkzeug der statistischen Datenanalyse und Signalverarbeitung, welches multivariate Signale in ihre Quellkomponenten zerlegen kann. Obwohl das klassische ICA Modell sehr nützlich ist, gibt es viele Anwendungen, die Erweiterungen von ICA erfordern. In dieser Dissertation präsentieren wir neue Verfahren, die die Funktionalität von ICA erweitern: (1) Zuverlässigkeitsanalyse und Gruppierung von unabhängigen Komponenten durch Hinzufügen von Rauschen, (2) robuste und überbestimmte ('over-complete') ICA durch Ausreissererkennung, und (3) nichtlineare ICA mit Kernmethoden.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... -Les Laplacian eigenmaps dans [131], [132] ou [133]. ...
Thesis
Full-text available
La reconnaissance faciale est l'une des meilleures modalités biométriques pour des applications liées à l'identification ou l'authentification de personnes. En effet, c'est la modalité utilisée par les humains. Elle est non intrusive et socialement bien acceptée. Malheureusement, les visages humains sont semblables et offrent par conséquent une faible possibilité de distinction par rapport à d'autres modalités biométriques, comme par exemple, les empreintes digitales et l'iris. Par ailleurs, lorsqu'il s'agit d'images 2D de visages, les variations intra-classe, dues à des facteurs aussi divers que les changements des conditions d'éclairage, variation de cosmétiques et de pose, sont généralement supérieures aux variations inter classes, ce qui rend la reconnaissance faciale 2D peu fiable dans des conditions réelles. Récemment, les représentations 3D de visages ont été largement étudiées par la communauté scientifique pour pallier les problèmes non résolus dans la reconnaissance faciale 2D. Cette thèse est consacrée à la reconnaissance faciale robuste utilisant la fusion des données faciales 2D et 3D. Nous consacrons la première partie de notre étude à la vérification de visage uni-modale et multi-algorithmes de visage 2D. Tout d’abord, nous étudions plusieurs méthodes pour sélectionner les meilleurs systèmes d’authentification de visages. Ensuite, nous présentons la multi-modalité et les méthodes de fusion de scores pour les deux approches combinaison et classification. Enfin les méthodes de fusion de scores sont comparées sur la base de données des visages XM2VTS. Dans la deuxième partie nous proposons un algorithme automatique d'authentification du visage par la fusion de deux systèmes multimodaux (multi-algorithmes et multi-capteurs 2D + 3D). Tout d'abord, nous avons corrigé la rotation de la tête par l'algorithme ICP, puis présenté six méthodes d'extraction de caractéristiques locales (MSLBP, CSL proposée, Ondelettes de Gabor, LBP, LPQ et BSIF). La classification des caractéristiques est réalisée par le métrique cosinus après réduction d'espace par EFM, puis fusion au niveau des scores par un bon classificateur à deux classes SVM. Enfin, l'application est réalisée sur les bases de données CASIA 3D et Bosphorus. Dans la dernière partie, nous étudions la vérification de visage uni-modale 2D et 3D et multimodale (2D+3D) basée sur la fusion de l'information locale. L'étude comprend trois étapes principales (prétraitement, extraction de caractéristiques et classification). Dans la première étape, une phase de prétraitement est nécessaire. L’algorithme ICP est utilisé pour aligner tous les visages et l'approche PS est utilisée pour réduire l'influence de la variation de l'éclairage pour les images 2D. Dans la deuxième étape nous avons utilisé quatre descripteurs locaux (LBP, LPQ, BSIF et Statistical LBP proposée). Après extraction des caractéristiques, l'image faciale 2D ou 3D est divisée en 10 régions et chaque région est divisée en 15 petites blocs. L'extraction de caractéristiques locales est résumée par les histogrammes correspondants de chaque bloc. Dans la dernière étape, nous proposons d'utiliser EDA couplée à WCCN pour la réduction de dimension des histogrammes pour chaque région. Nous validons nos méthodes proposées en les comparant avec celles existant dans la littérature scientifique sur les bases de données FRGC v2 et CASIA 3D.
... Elle permet également souvent de limiter l'effet de la malédiction de la dimension et d'autres propriétés non désirées des espaces de grande dimension [152]. Récemment, un grand nombre de méthodes de réduction de dimension ont été proposées [35], [82], [135], [172], [246], [255], [272], [275], [306][138], la factorisation non–négative de matrices (NMF) [48], l'Analyse en Composantes Indépendantes (ICA) [127] et l'Analyse Discriminante Linéaire (LDA) [107], ainsi que dix méthodes non–linéaires (le nom de chaque méthode n'a volontairement pas été traduit) : multidimensional scaling (MDS) [71], [166], Isomap [274], [275], Kernel PCA [204], [255], diffusion maps [172], [214] , multilayer autoencoders [78], [135], Locally Linear Embedding (LLE) [246], Laplacian [20]. ...
Article
Full-text available
In this thesis, we describe the problem of automatic face recognition in visibleand long–wave infrared lights. The state of the art methods are described, and westudy, in a first time, a method based on convolutional neural networks. Applied toboth modalities, the proposed fusion method, based on a weighted sum of scores,yields a substantial increasing of the recognition rates. In a second time, a pretrainingof the network with sparse methods is studied for automatic facial recognition.Finally, we propose an approach based on a sparse decomposition of faces, coupledwith a classification scheme involving in a l1 minimization. This last approach givesgood identification rates on the well known Notre–Dame database.
... Inlier-based ICA (ibica) by Harmeling, Meinecke and Müller, Fraunhofer FIRST.IDA, Berlin, 2004, refer to [4], [5]. Ibica also is a geometric algorithm. ...
Article
Full-text available
The Relaxation to have only square matrices in standard ICA leads to an approximation of the inverse of non-quadratic matrices to determine the separation matrix. Synthetic data sets as well as speech data are used to compare the capability of such approaches, called overcomplete ICA on an underdetermined basis. Due to the fact that the mixing matrices are not invertible (because they are not square), the quality of the sources' reconstruction is not excellent. The most extreme case of an undetermined ICA is single channel ICA. But in this paper not the reduction to one sensor is considered but in a maximum case the reduction from eight sensors to two sensor signals. It is shown, which separation quality can still be achieved f or the blind separation of the underlying sources. For an improved classification the algorithms are also compared to well-known standard ICA-algorithms.
... [96]) called KPCA) and ICA (cf. [60]). ...
Article
The goal of a Brain-Computer Interface (BCI) consists of the development of a unidirectional interface between a human and a computer to allow control of a device only via brain signals. While the BCI systems of almost all other groups require the user to be trained over several weeks or even months, the group of Prof. Dr. Klaus-Robert Müller in Berlin and Potsdam, which I belong to, was one of the first research groups in this field which used machine learning techniques on a large scale. The adaptivity of the processing system to the individual brain patterns of the subject confers huge advantages for the user. Thus BCI research is considered a hot topic in machine learning and computer science. It requires interdisciplinary cooperation between disparate fields such as neuroscience, since only by combining machine learning and signal processing techniques based on neurophysiological knowledge will the largest progress be made. In this work I particularly deal with my part of this project, which lies mainly in the area of computer science. I have considered the following three main points: Establishing a performance measure based on information theory: I have critically illuminated the assumptions of Shannon's information transfer rate for application in a BCI context. By establishing suitable coding strategies I was able to show that this theoretical measure approximates quite well to what is practically achieveable. Transfer and development of suitable signal processing and machine learning techniques: One substantial component of my work was to develop several machine learning and signal processing algorithms to improve the efficiency of a BCI. Based on the neurophysiological knowledge that several independent EEG features can be observed for some mental states, I have developed a method for combining different and maybe independent features which improved performance. In some cases the performance of the combination algorithm outperforms the best single performance by more than 50 %. Furthermore, I have theoretically and practically addressed via the development of suitable algorithms the question of the optimal number of classes which should be used for a BCI. It transpired that with BCI performances reported so far, three or four different mental states are optimal. For another extension I have combined ideas from signal processing with those of machine learning since a high gain can be achieved if the temporal filtering, i.e., the choice of frequency bands, is automatically adapted to each subject individually. Implementation of the Berlin brain computer interface and realization of suitable experiments: Finally a further substantial component of my work was to realize an online BCI system which includes the developed methods, but is also flexible enough to allow the simple realization of new algorithms and ideas. So far, bitrates of up to 40 bits per minute have been achieved with this system by absolutely untrained users which, compared to results of other groups, is highly successful.
... There are two aspects in dimension reduction of process dynamic trends: feature extraction from a trend of an individual variable and removal of dependencies among a number of correlated and sometimes redundant variables [3]. Independent component analysis (ICA) aims at extracting unknown hidden factor/components from multivariate data using only the assumption that the unknown factors are mutually independent [4]. Rough sets are a tool to deal with inexact, uncertain or vague knowledge. ...
Conference Paper
Full-text available
Web Usage Mining is that area of Web Mining which deals with the extraction of interesting knowledge from logging information produced by Web servers. A challenge in web classification is how to deal with the high dimensionality of the feature space. In this paper we present Independent Component Analysis (ICA) for feature selection and using Rough Fuzzy for clustering web user sessions. It aims at discovery of trends and regularities in web users' access patterns. ICA is a very general-purpose statistical technique in which observed random data are linearly transformed into components that are maximally independent from each other, and simultaneously have "interesting" distributions. Our experiments indicate can improve the predictive performance when the original feature set for representing web log is large and can handling the different groups of uncertainties/impreciseness accuracy.
Chapter
Full-text available
We give an overview of recent advances in cognitive-based neural prostheses, and point out the major differences with respect to commonly used motor-based brain-machine interfaces. While encouraging results in neuroprosthetic research have demonstrated the proof of concept, the development of practical neural prostheses is still in the phase of infancy. To address complex issues arising in the development of practical neural prostheses we review several related studies ranging from the identification of new cognitive variables to the development of novel signal processing tools. In the second part of this chapter, we discuss an information-theoretic approach to the extraction of low-dimensional features from high-dimensional neural data. We argue that this approach may be better suited for certain neuroprosthetic applications than the traditionally used features. An extensive analysis of electrical recordings from the human
Article
Full-text available
19.1 Abstract To analyze the performance of BCI systems, some evaluation criteria must be applied. The most popular is accuracy or error rate. Because of some strict prerequisites, accuracy is not always a suitable criterion, and other evaluation criteria have been proposed. This chapter provides an overview of evaluation criteria used in BCI research. An example from the BCI Competition 2005 is used to display results using different criteria. Within this chapter, evaluation criteria for BCI systems with more than two classes are presented, criteria for evaluating discrete and continuous output are included, and the problem of evaluating self-paced BCI operation is addressed. Special emphasis is put on discussing different methods for calculating the information transfer rate. Finally, a criterion for taking into account the response time is suggested.
Article
This paper proposes a new independent component analysis (ICA) method which is able to unmix overcomplete mixtures of sparce or structured signals like speech, music or images. Furthermore, the method is designed to be robust against outliers, which is a favorable feature for ICA algorithms since most of them are extremely sensitive to outliers. Our approach is based on a simple outlier index. However, instead of robustifying an existing algorithm by some outlier rejection technique we show how this index can be used directly to solve the ICA problem for super-Gaussian sources. The resulting inlier-based ICA (IBICA) is outlier-robust by construction and can be used for standard ICA as well as for overcomplete ICA (i.e. more source signals than observed signals). © 2005 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 15, 48–55, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.20037
Article
We propose simple and fast methods based on nearest neighbors that order objects from high-dimensional data sets from typical points to untypical points. On the one hand, we show that these easy-to-compute orderings allow us to detect outliers (i.e. very untypical points) with a performance comparable to or better than other often much more sophisticated methods. On the other hand, we show how to use these orderings to detect prototypes (very typical points) which facilitate exploratory data analysis algorithms such as noisy nonlinear dimensionality reduction and clustering. Comprehensive experiments demonstrate the validity of our approach.
ResearchGate has not been able to resolve any references for this publication.