-
[show abstract]
[hide abstract]
ABSTRACT: Properties of data are frequently seen to vary depending on the sampled situations, which usually change along a time evolution or owing to environmental effects. One way to analyze such data is to find invariances, or representative features kept constant over changes. The aim of this paper is to identify one such feature, namely interactions or dependencies among variables that are common across multiple datasets collected under different conditions. To that end, we propose a common substructure learning (CSSL) framework based on a graphical Gaussian model. We further present a simple learning algorithm based on the Dual Augmented Lagrangian and the Alternating Direction Method of Multipliers. We confirm the performance of CSSL over other existing techniques in finding unchanging dependency structures in multiple datasets through numerical simulations on synthetic data and through a real world application to anomaly detection in automobile sensors.
Neural networks: the official journal of the International Neural Network Society 11/2012; 38C:23-38. · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Non-stationary effects are ubiquitous in real world data. In many settings, the observed signals are a mixture of underlying stationary and non-stationary sources that cannot be measured directly. For example, in EEG analysis, electrodes on the scalp record the activity from several sources located inside the brain, which one could only measure invasively. Discerning stationary and non-stationary contributions is an important step towards uncovering the mechanisms of the data generating system. To that end, in Stationary Subspace Analysis (SSA), the observed signal is modeled as a linear superposition of stationary and non-stationary sources, where the aim is to separate the two groups in the mixture. In this paper, we propose the first SSA algorithm that has a closed form solution. The novel method, Analytic SSA (ASSA), is more than 100 times faster than the state-of-the-art, numerically stable, and guaranteed to be optimal when the covariance between stationary and non-stationary sources is time-constant. In numerical simulations on wide range of settings, we show that our method yields superior results, even for signals with time-varying group-wise covariance. In an application to geophysical data analysis, ASSA extracts meaningful components that shed new light on the Pi 2 pulsations of the geomagnetic field.
Neural networks: the official journal of the International Neural Network Society 04/2012; 33:7-20. · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We consider to learn a causal ordering of variables in a linear non-Gaussian
acyclic model called LiNGAM. Several existing methods have been shown to
consistently estimate a causal ordering assuming that all the model assumptions
are correct. But, the estimation results could be distorted if some assumptions
actually are violated. In this paper, we propose a new algorithm for learning
causal orders that is robust against one typical violation of the model
assumptions: latent confounders. We demonstrate the effectiveness of our method
using artificial data.
04/2012;
-
[show abstract]
[hide abstract]
ABSTRACT: Discovering causal relations among observed variables in a given data set is
a main topic in studies of statistics and artificial intelligence. Recently,
some techniques to discover an identifiable causal structure have been explored
based on non-Gaussianity of the observed data distribution. However, most of
these are limited to continuous data. In this paper, we present a novel causal
model for binary data and propose a new approach to derive an identifiable
causal structure governing the data based on skew Bernoulli distributions of
external noise. Experimental evaluation shows excellent performance for both
artificial and real world data sets.
02/2012;
-
[show abstract]
[hide abstract]
ABSTRACT: In this paper, we propose the first exact algorithm for minimizing the
difference of two submodular functions (D.S.), i.e., the discrete version of
the D.C. programming problem. The developed algorithm is a
branch-and-bound-based algorithm which responds to the structure of this
problem through the relationship between submodularity and convexity. The D.S.
programming problem covers a broad range of applications in machine learning
because this generalizes the optimization of a wide class of set functions. We
empirically investigate the performance of our algorithm, and illustrate the
difference between exact and approximate solutions respectively obtained by the
proposed and existing algorithms in feature selection and discriminative
structure learning.
08/2011;
-
[show abstract]
[hide abstract]
ABSTRACT: Learning underlying mechanisms of data generation is of great interest in the scientific and engineering fields amongst others.
Finding dependency structures among variables in the data is one possible approach for the purpose, and is an important task
in data mining. In this paper, we focus on learning dependency substructures shared by multiple datasets. In many scenarios,
the nature of data varies due to a change in the surrounding conditions or non-stationary mechanisms over the multiple datasets.
However, we can also assume that the change occurs only partially and some relations between variables remain unchanged. Moreover,
we can expect that such commonness over the multiple datasets is closely related to the invariance of the underlying mechanism.
For example, errors in engineering systems are usually caused by faults in the sub-systems with the other parts remaining
healthy. In such situations, though anomalies are observed in sensor values, the underlying invariance of the healthy sub-systems
is still captured by some steady dependency structures before and after the onset of the error. We propose a structure learning
algorithm to find such invariances in the case of Graphical Gaussian Models (GGM). The proposed method is based on a block
coordinate descent optimization, where subproblems can be solved efficiently by existing algorithms for Lasso and the continuous quadratic knapsack problem. We confirm the validity of our approach through numerical simulations and also in applications with real world datasets
extracted from the analysis of city-cycle fuel consumption and anomaly detection in car sensors.
KeywordsGraphical Gaussian Model–common substructure–block coordinate descent
08/2011: pages 1-16;
-
[show abstract]
[hide abstract]
ABSTRACT: Many statistical methods have been proposed to estimate causal models in classical situations with fewer variables than observations. However, modern datasets including gene expression data increase the needs of high-dimensional causal modeling in challenging situations with orders of magnitude more variables than observations. In this paper, we propose a method to find exogenous variables in a linear non-Gaussian causal model, which requires much smaller sample sizes than conventional methods and works even under orders of magnitude more variables than observations. Exogenous variables work as triggers that activate causal chains in the model, and their identification leads to more efficient experimental designs and better understanding of the causal mechanism. We present experiments with artificial data and real-world gene expression data to evaluate the method.
Neural networks: the official journal of the International Neural Network Society 06/2011; 24(8):875-80. · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Structural equation models and Bayesian networks have been widely used to
analyze causal relations between continuous variables. In such frameworks,
linear acyclic models are typically used to model the data-generating process
of variables. Recently, it was shown that use of non-Gaussianity identifies the
full structure of a linear acyclic model, i.e., a causal ordering of variables
and their connection strengths, without using any prior knowledge on the
network structure, which is not the case with conventional methods. However,
existing estimation methods are based on iterative search algorithms and may
not converge to a correct solution in a finite number of steps. In this paper,
we propose a new direct method to estimate a causal ordering and connection
strengths based on non-Gaussianity.
In contrast to the previous methods, our algorithm requires no algorithmic
parameters and is guaranteed to converge to the right solution within a small
fixed number of steps if the data strictly follows the model.
01/2011;
-
Neurocomputing. 01/2011; 74:2212-2221.
-
UAI 2011, Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, Barcelona, Spain, July 14-17, 2011; 01/2011
-
[show abstract]
[hide abstract]
ABSTRACT: Finding the structure of a graphical model has been received much attention in many fields. Recently, it is reported that the non-Gaussianity of data enables us to identify the structure of a directed acyclic graph without any prior knowledge on the structure. In this paper, we propose a novel non-Gaussianity based algorithm for more general type of models; chain graphs. The algorithm finds an ordering of the disjoint subsets of variables by iteratively evaluating the independence between the variable subset and the residuals when the remaining variables are regressed on those. However, its computational cost grows exponentially according to the number of variables. Therefore, we further discuss an efficient approximate approach for applying the algorithm to large sized graphs. We illustrate the algorithm with artificial and real-world datasets.
06/2010;
-
Artificial Neural Networks - ICANN 2010 - 20th International Conference, Thessaloniki, Greece, September 15-18, 2010, Proceedings, Part I; 01/2010
-
International Joint Conference on Neural Networks, IJCNN 2010, Barcelona, Spain, 18-23 July, 2010; 01/2010
-
Neural Information Processing. Theory and Algorithms - 17th International Conference, ICONIP 2010, Sydney, Australia, November 22-25, 2010, Proceedings, Part I; 01/2010
-
Latent Variable Analysis and Signal Separation - 9th International Conference, LVA/ICA 2010, St. Malo, France, September 27-30, 2010. Proceedings; 01/2010
-
[show abstract]
[hide abstract]
ABSTRACT: Many statistical methods have been proposed to estimate causal models in
classical situations with fewer variables than observations (p<n, p: the number
of variables and n: the number of observations). However, modern datasets
including gene expression data need high-dimensional causal modeling in
challenging situations with orders of magnitude more variables than
observations (p>>n). In this paper, we propose a method to find exogenous
variables in a linear non-Gaussian causal model, which requires much smaller
sample sizes than conventional methods and works even when p>>n. The key idea
is to identify which variables are exogenous based on non-Gaussianity instead
of estimating the entire structure of the model. Exogenous variables work as
triggers that activate a causal chain in the model, and their identification
leads to more efficient experimental designs and better understanding of the
causal mechanism. We present experiments with artificial data and real-world
gene expression data to evaluate the method.
04/2009;
