Klaus-Robert Müller

Publications of Klaus-Robert Müller

• Optimizing transition states via kernel-based machine learning.

  Authors: Zachary D Pozun, Katja Hansen, Daniel Sheppard, Matthias Rupp, Klaus-Robert Müller, Graeme Henkelman

  The Journal of chemical physics. 05/2012; 136(17):174101.

  We present a method for optimizing transition state theory dividing surfaces with support vector machines. The resulting dividing surfaces require no a priori information or intuition about reactionWe present a method for optimizing transition state theory dividing surfaces with support vector machines. The resulting dividing surfaces require no a priori information or intuition about reaction mechanisms. To generate optimal dividing surfaces, we apply a cycle of machine-learning and refinement of the surface by molecular dynamics sampling. We demonstrate that the machine-learned surfaces contain the relevant low-energy saddle points. The mechanisms of reactions may be extracted from the machine-learned surfaces in order to identify unexpected chemically relevant processes. Furthermore, we show that the machine-learned surfaces significantly increase the transmission coefficient for an adatom exchange involving many coupled degrees of freedom on a (100) surface when compared to a distance-based dividing surface.

• Improved decoding of neural activity from fMRI signals using non-separable spatiotemporal deconvolutions.

  Authors: Felix Bießmann, Yusuke Murayama, Nikos K Logothetis, Klaus-Robert Müller, Frank C Meinecke

  NeuroImage. 04/2012;

  The goal of most functional Magnetic Resonance Imaging (fMRI) analyses is to investigate neural activity. Many fMRI analysis methods assume that the temporal dynamics of the hemodynamic responseThe goal of most functional Magnetic Resonance Imaging (fMRI) analyses is to investigate neural activity. Many fMRI analysis methods assume that the temporal dynamics of the hemodynamic response function (HRF) to neural activation is separable from its spatial dynamics. Although there is empirical evidence that the HRF is more complex than suggested by space-time separable canonical HRF models, it is difficult to assess how much information about neural activity is lost when assuming space-time separability. In this study we directly test whether spatiotemporal variability in the HRF that is not captured by separable models contains information about neural signals. We predict intracranially measured neural activity from simultaneously recorded fMRI data using separable and non-separable spatiotemporal deconvolutions of voxel time series around the recording electrode. Our results show that abandoning the spatiotemporal separability assumption consistently improves the decoding accuracy of neural signals from fMRI data. We compare our findings with results from optical imaging and fMRI studies and discuss potential implications for classical fMRI analyses without invasive electrophysiological recordings.

• Stationary common spatial patterns for brain-computer interfacing.

  Authors: Wojciech Samek, Carmen Vidaurre, Klaus-Robert Müller, Motoaki Kawanabe

  Journal of neural engineering. 02/2012; 9(2):026013.

  Classifying motion intentions in brain-computer interfacing (BCI) is a demanding task as the recorded EEG signal is not only noisy and has limited spatial resolution but it is also intrinsicallyClassifying motion intentions in brain-computer interfacing (BCI) is a demanding task as the recorded EEG signal is not only noisy and has limited spatial resolution but it is also intrinsically non-stationary. The non-stationarities in the signal may come from many different sources, for instance, electrode artefacts, muscular activity or changes of task involvement, and often deteriorate classification performance. This is mainly because features extracted by standard methods like common spatial patterns (CSP) are not invariant to variations of the signal properties, thus should also change over time. Although many extensions of CSP were proposed to, for example, reduce the sensitivity to noise or incorporate information from other subjects, none of them tackles the non-stationarity problem directly. In this paper, we propose a method which regularizes CSP towards stationary subspaces (sCSP) and show that this increases classification accuracy, especially for subjects who are hardly able to control a BCI. We compare our method with the state-of-the-art approaches on different datasets, show competitive results and analyse the reasons for the improvement.

• An Algebraic Method for Approximate Rank One Factorization of Rank Deficient Matrices.

  Authors: Franz J. Király, Andreas Ziehe, Klaus-Robert Müller

  Latent Variable Analysis and Signal Separation - 10th International Conference, LVA/ICA 2012, Tel Aviv, Israel, March 12-15, 2012. Proceedings; 01/2012

• Finding Density Functionals with Machine Learning

  Authors: John C. Snyder, Matthias Rupp, Katja Hansen, Klaus-Robert Müller, Kieron Burke

  12/2011;

  Machine learning is used to approximate density functionals. For the model problem of the kinetic energy of non-interacting fermions in 1d, mean absolute errors below 1 kcal/mol on test densitiesMachine learning is used to approximate density functionals. For the model problem of the kinetic energy of non-interacting fermions in 1d, mean absolute errors below 1 kcal/mol on test densities similar to the training set are reached with fewer than 100 training densities. A predictor identifies if a test density is within the interpolation region. Via principal component analysis, a projected functional derivative finds highly accurate self-consistent densities. Challenges for application of our method to real electronic structure problems are discussed.

• Insights from Classifying Visual Concepts with Multiple Kernel Learning

  Authors: Alexander Binder, Shinichi Nakajima, Marius Kloft, Christina Müller, Wojciech Samek, Ulf Brefeld, Klaus-Robert Müller, Motoaki Kawanabe

  12/2011;

  Combining information from various image features has become a standard technique in concept recognition tasks. However, the optimal way of fusing the resulting kernel functions is usually unknown inCombining information from various image features has become a standard technique in concept recognition tasks. However, the optimal way of fusing the resulting kernel functions is usually unknown in practical applications. Multiple kernel learning (MKL) techniques allow to determine an optimal linear combination of such similarity matrices. Classical approaches to MKL promote sparse mixtures. Unfortunately, so-called 1-norm MKL variants are often observed to be outperformed by an unweighted sum kernel. The contribution of this paper is twofold: We apply a recently developed non-sparse MKL variant to state-of-the-art concept recognition tasks within computer vision. We provide insights on benefits and limits of non-sparse MKL and compare it against its direct competitors, the sum kernel SVM and the sparse MKL. We report empirical results for the PASCAL VOC 2009 Classification and ImageCLEF2010 Photo Annotation challenge data sets. About to be submitted to PLoS ONE.

• Regression for sets of polynomial equations

  Authors: Franz Johannes Király, Paul von Bünau, Jan Saputra Müller, Duncan Blythe, Frank Meinecke, Klaus-Robert Müller

  10/2011;

  We propose a method called ideal regression for approximating an arbitrary system of polynomial equations by a system of a particular type. Using techniques from approximate computational algebraicWe propose a method called ideal regression for approximating an arbitrary system of polynomial equations by a system of a particular type. Using techniques from approximate computational algebraic geometry, we show how we can solve ideal regression directly without resorting to numerical optimization. Ideal regression is useful whenever the solution to a learning problem can be described by a system of polynomial equations. As an example, we demonstrate how to formulate Stationary Subspace Analysis (SSA), a source separation problem, in terms of ideal regression, which also yields a consistent estimator for SSA. We then compare this estimator in simulations with previous optimization-based approaches for SSA.

• Psychological predictors of SMR-BCI performance.

  Authors: Eva Maria Hammer, Sebastian Halder, Benjamin Blankertz, Claudia Sannelli, Thorsten Dickhaus, Sonja Kleih, Klaus-Robert Müller, Andrea Kübler

  Biological psychology. 09/2011; 89(1):80-6.

  After about 30 years of research on Brain-Computer Interfaces (BCIs) there is little knowledge about the phenomenon, that some people - healthy as well as individuals with disease - are not able toAfter about 30 years of research on Brain-Computer Interfaces (BCIs) there is little knowledge about the phenomenon, that some people - healthy as well as individuals with disease - are not able to learn BCI-control. To elucidate this "BCI-inefficiency" phenomenon, the current study investigated whether psychological parameters, such as attention span, personality or motivation, could predict performance in a single session with a BCI controlled by modulation of sensorimotor rhythms (SMR) with motor imagery. A total of N=83 healthy BCI novices took part in the session. Psychological parameters were measured with an electronic test-battery including clinical, personality and performance tests. Predictors were determined by binary logistic regression analyses. The output variable of the Two-Hand Coordination Test (2HAND) "overall mean error duration" which is a measure for the accuracy of fine motor skills accounted for 11% of the variance in BCI-inefficiency. The Attitudes Towards Work (AHA) test variable "performance level" which can be interpreted as a degree of concentration and a neurophysiological SMR predictor were also identified as significant predictors of SMR BCI performance. Psychological parameters as measured in this study play a moderate role for one-session performance in a BCI controlled by modulation of SMR.

• Directional Variance Adjustment: improving covariance estimates for high-dimensional portfolio optimization

  Authors: Daniel Bartz, Kerr Hatrick, Christian W. Hesse, Klaus-Robert Müller, Steven Lemm

  09/2011;

  Robust and reliable covariance estimates play a decisive role in financial and many other applications. An important class of estimators is based on Factor models. Here, we show by extensive MonteRobust and reliable covariance estimates play a decisive role in financial and many other applications. An important class of estimators is based on Factor models. Here, we show by extensive Monte Carlo simulations that covariance matrices derived from the statistical Factor Analysis model exhibit a systematic error, which is similar to the well-known systematic error of the spectrum of the sample covariance matrix. Moreover, we introduce the Directional Variance Adjustment (DVA) algorithm, which diminishes the systematic error. In a thorough empirical study for the US, European, and Hong Kong market we show that our proposed method leads to improved portfolio allocation.

• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning

  Authors: Matthias Rupp, Alexandre Tkatchenko, Klaus-Robert Müller, O. Anatole von Lilienfeld

  09/2011;

  We introduce a machine learning model to predict atomization energies of a diverse set of organic molecules, based on nuclear charges and atomic positions only. The problem of solving the molecularWe introduce a machine learning model to predict atomization energies of a diverse set of organic molecules, based on nuclear charges and atomic positions only. The problem of solving the molecular Schr\"odinger equation is mapped onto a non-linear statistical regression problem of reduced complexity. Regression models are trained on and compared to atomization energies computed with hybrid density-functional theory. Cross-validation over more than seven thousand small organic molecules yields a mean absolute error of ~10 kcal/mol. Applicability is demonstrated for the prediction of molecular atomization potential energy curves.

• Feature Extraction for Change-Point Detection using Stationary Subspace Analysis

  Authors: Duncan Blythe, Paul von Bünau, Frank Meinecke, Klaus-Robert Müller

  08/2011;

  Detecting changes in high-dimensional time series is difficult because it involves the comparison of probability densities that need to be estimated from finite samples. In this paper, we present theDetecting changes in high-dimensional time series is difficult because it involves the comparison of probability densities that need to be estimated from finite samples. In this paper, we present the first feature extraction method tailored to change point detection, which is based on an extended version of Stationary Subspace Analysis. We reduce the dimensionality of the data to the most non-stationary directions, which are most informative for detecting state changes in the time series. In extensive simulations on synthetic data we show that the accuracy of three change point detection algorithms is significantly increased by a prior feature extraction step. These findings are confirmed in an application to industrial fault monitoring.

• Enhanced performance by a hybrid NIRS-EEG brain computer interface.

  Authors: Siamac Fazli, Jan Mehnert, Jens Steinbrink, Gabriel Curio, Arno Villringer, Klaus-Robert Müller, Benjamin Blankertz

  NeuroImage. 08/2011; 59(1):519-29.

  Noninvasive Brain Computer Interfaces (BCI) have been promoted to be used for neuroprosthetics. However, reports on applications with electroencephalography (EEG) show a demand for a better accuracyNoninvasive Brain Computer Interfaces (BCI) have been promoted to be used for neuroprosthetics. However, reports on applications with electroencephalography (EEG) show a demand for a better accuracy and stability. Here we investigate whether near-infrared spectroscopy (NIRS) can be used to enhance the EEG approach. In our study both methods were applied simultaneously in a real-time Sensory Motor Rhythm (SMR)-based BCI paradigm, involving executed movements as well as motor imagery. We tested how the classification of NIRS data can complement ongoing real-time EEG classification. Our results show that simultaneous measurements of NIRS and EEG can significantly improve the classification accuracy of motor imagery in over 90% of considered subjects and increases performance by 5% on average (p<0:01). However, the long time delay of the hemodynamic response may hinder an overall increase of bit-rates. Furthermore we find that EEG and NIRS complement each other in terms of information content and are thus a viable multimodal imaging technique, suitable for BCI.

• Single-trial analysis and classification of ERP components--a tutorial.

  Authors: Benjamin Blankertz, Steven Lemm, Matthias Treder, Stefan Haufe, Klaus-Robert Müller

  NeuroImage. 05/2011; 56(2):814-25.

  Analyzing brain states that correspond to event related potentials (ERPs) on a single trial basis is a hard problem due to the high trial-to-trial variability and the unfavorable ratio between signalAnalyzing brain states that correspond to event related potentials (ERPs) on a single trial basis is a hard problem due to the high trial-to-trial variability and the unfavorable ratio between signal (ERP) and noise (artifacts and neural background activity). In this tutorial, we provide a comprehensive framework for decoding ERPs, elaborating on linear concepts, namely spatio-temporal patterns and filters as well as linear ERP classification. However, the bottleneck of these techniques is that they require an accurate covariance matrix estimation in high dimensional sensor spaces which is a highly intricate problem. As a remedy, we propose to use shrinkage estimators and show that appropriate regularization of linear discriminant analysis (LDA) by shrinkage yields excellent results for single-trial ERP classification that are far superior to classical LDA classification. Furthermore, we give practical hints on the interpretation of what classifiers learned from the data and demonstrate in particular that the trade-off between goodness-of-fit and model complexity in regularized LDA relates to a morphing between a difference pattern of ERPs and a spatial filter which cancels non task-related brain activity.

• Introduction to machine learning for brain imaging.

  Authors: Steven Lemm, Benjamin Blankertz, Thorsten Dickhaus, Klaus-Robert Müller

  NeuroImage. 05/2011; 56(2):387-99.

  Machine learning and pattern recognition algorithms have in the past years developed to become a working horse in brain imaging and the computational neurosciences, as they are instrumental forMachine learning and pattern recognition algorithms have in the past years developed to become a working horse in brain imaging and the computational neurosciences, as they are instrumental for mining vast amounts of neural data of ever increasing measurement precision and detecting minuscule signals from an overwhelming noise floor. They provide the means to decode and characterize task relevant brain states and to distinguish them from non-informative brain signals. While undoubtedly this machinery has helped to gain novel biological insights, it also holds the danger of potential unintentional abuse. Ideally machine learning techniques should be usable for any non-expert, however, unfortunately they are typically not. Overfitting and other pitfalls may occur and lead to spurious and nonsensical interpretation. The goal of this review is therefore to provide an accessible and clear introduction to the strengths and also the inherent dangers of machine learning usage in the neurosciences.

• ℓ(1)-penalized linear mixed-effects models for high dimensional data with application to BCI.

  Authors: Siamac Fazli, Márton Danóczy, Jürg Schelldorfer, Klaus-Robert Müller

  NeuroImage. 04/2011; 56(4):2100-8.

  Recently, a novel statistical model has been proposed to estimate population effects and individual variability between subgroups simultaneously, by extending Lasso methods. We will for the firstRecently, a novel statistical model has been proposed to estimate population effects and individual variability between subgroups simultaneously, by extending Lasso methods. We will for the first time apply this so-called ℓ(1)-penalized linear regression mixed-effects model for a large scale real world problem: we study a large set of brain computer interface data and through the novel estimator are able to obtain a subject-independent classifier that compares favorably with prior zero-training algorithms. This unifying model inherently compensates shifts in the input space attributed to the individuality of a subject. In particular we are now for the first time able to differentiate within-subject and between-subject variability. Thus a deeper understanding both of the underlying statistical and physiological structures of the data is gained.

• Co-adaptive calibration to improve BCI efficiency.

  Authors: Carmen Vidaurre, Claudia Sannelli, Klaus-Robert Müller, Benjamin Blankertz

  Journal of neural engineering. 03/2011; 8(2):025009.

  All brain-computer interface (BCI) groups that have published results of studies involving a large number of users performing BCI control based on the voluntary modulation of sensorimotor rhythmsAll brain-computer interface (BCI) groups that have published results of studies involving a large number of users performing BCI control based on the voluntary modulation of sensorimotor rhythms (SMR) report that BCI control could not be achieved by a non-negligible number of subjects (estimated 20% to 25%). This failure of the BCI system to read the intention of the user is one of the greatest problems and challenges in BCI research. There are two main causes for this problem in SMR-based BCI systems: either no idle SMR is observed over motor areas of the user, or this idle rhythm is not modulated during motor imagery, resulting in a classification performance lower than 70% (criterion level) that renders the control of a BCI application (like a speller) difficult or impossible. Previously, we introduced the concept of machine learning based co-adaptive calibration, which provided substantially improved performance for a variety of users. Here, we use a similar approach and investigate to what extent co-adaptive learning enables significant BCI control for completely novice users, as well as for those who could not achieve control with a conventional SMR-based BCI.

• CSP patches: an ensemble of optimized spatial filters. An evaluation study.

  Authors: Claudia Sannelli, Carmen Vidaurre, Klaus-Robert Müller, Benjamin Blankertz

  Journal of neural engineering. 03/2011; 8(2):025012.

  Laplacian filters are widely used in neuroscience. In the context of brain-computer interfacing, they might be preferred to data-driven approaches such as common spatial patterns (CSP) in a varietyLaplacian filters are widely used in neuroscience. In the context of brain-computer interfacing, they might be preferred to data-driven approaches such as common spatial patterns (CSP) in a variety of scenarios such as, e.g., when no or few user data are available or a calibration session with a multi-channel recording is not possible, which is the case in various applications. In this paper we propose the use of an ensemble of local CSP patches (CSPP) which can be considered as a compromise between Laplacian filters and CSP. Our CSPP only needs a very small number of trials to be optimized and significantly outperforms Laplacian filters in all settings studied. Additionally, CSPP also outperforms multi-channel CSP and a regularized version of CSP even when only very few calibration data are available, acting as a CSP regularizer without the need of additional hyperparameters and at a very low cost: 2-5 min of data recording, i.e. ten times less than CSP.

• Large-scale EEG/MEG source localization with spatial flexibility.

  Authors: Stefan Haufe, Ryota Tomioka, Thorsten Dickhaus, Claudia Sannelli, Benjamin Blankertz, Guido Nolte, Klaus-Robert Müller

  NeuroImage. 01/2011; 54(2):851-9.

  We propose a novel approach to solving the electro-/magnetoencephalographic (EEG/MEG) inverse problem which is based upon a decomposition of the current density into a small number of spatial basisWe propose a novel approach to solving the electro-/magnetoencephalographic (EEG/MEG) inverse problem which is based upon a decomposition of the current density into a small number of spatial basis fields. It is designed to recover multiple sources of possibly different extent and depth, while being invariant with respect to phase angles and rotations of the coordinate system. We demonstrate the method's ability to reconstruct simulated sources of random shape and show that the accuracy of the recovered sources can be increased, when interrelated field patterns are co-localized. Technically, this leads to large-scale mathematical problems, which are solved using recent advances in convex optimization. We apply our method for localizing brain areas involved in different types of motor imagery using real data from Brain-Computer Interface (BCI) sessions. Our approach based on single-trial localization of complex Fourier coefficients yields class-specific focal sources in the sensorimotor cortices.

• Analysis of multimodal neuroimaging data.

  Authors: Felix Biessmann, Sergey Plis, Frank C Meinecke, Tom Eichele, Klaus-Robert Müller

  IEEE reviews in biomedical engineering. 01/2011; 4:26-58.

  Each method for imaging brain activity has technical or physiological limits. Thus, combinations of neuroimaging modalities that can alleviate these limitations such as simultaneous recordings ofEach method for imaging brain activity has technical or physiological limits. Thus, combinations of neuroimaging modalities that can alleviate these limitations such as simultaneous recordings of neurophysiological and hemodynamic activity have become increasingly popular. Multimodal imaging setups can take advantage of complementary views on neural activity and enhance our understanding about how neural information processing is reflected in each modality. However, dedicated analysis methods are needed to exploit the potential of multimodal methods. Many solutions to this data integration problem have been proposed, which often renders both comparisons of results and the choice of the right method for the data at hand difficult. In this review we will discuss different multimodal neuroimaging setups, the advances achieved in basic research and clinical application and the methods used. We will provide a comprehensive overview of mathematical tools reoccurring in multimodal neuroimaging studies for artifact removal, data-driven and model-driven analyses, enabling the practitioner to try established or new combinations from these algorithmic building blocks.

• Uniqueness of Non-Gaussianity-Based Dimension Reduction.

  Authors: Fabian J. Theis, Motoaki Kawanabe, Klaus-Robert Müller

  IEEE Transactions on Signal Processing. 01/2011; 59:4478-4482.