Conference Paper

An Efficient Incremental Kernel Principal Component Analysis for Online Feature Selection.

Kobe Univ., Kobe
DOI: 10.1109/IJCNN.2007.4371325 Conference: Proceedings of the International Joint Conference on Neural Networks, IJCNN 2007, Celebrating 20 years of neural networks, Orlando, Florida, USA, August 12-17, 2007
Source: DBLP

ABSTRACT In this paper, a feature extraction method for online classification problems is proposed by extending kernel principal component analysis (KPCA). In our previous work, we proposed an incremental KPCA algorithm which could learn a new input incrementally without keeping all the past training data. In this algorithm, eigenvectors are represented by a linear sum of linearly independent data which are selected from given training data. A serious drawback of the previous IKPCA is that many independent data are prone to be selected during learning and this causes large computation and memory costs. For this problem, we propose a novel approach to the selection of independent data; that is, they are not selected in the high-dimensional feature space but in the low-dimensional eigenspace spanned by the current eigenvectors. Using this method, the number of independent data is restricted to the number of eigenvectors. This restriction makes the learning of the modified IKPCA (M-IKPCA) very fast without loosing the approximation accuracy against true eigenvectors. To verify the effectiveness of M-IKPCA, the learning time and the accuracy of eigenspaces are evaluated using two UCI benchmark datasets. As a result, we confirm that the learning of M-IKPCA is at least 5 times faster than the previous version of IKPCA.

  • [Show abstract] [Hide abstract]
    ABSTRACT: In this paper, a new incremental learning algo­ rithm of Kernel Principal Component Analysis (KPCA) is proposed for online feature extraction in pattern recognition problems. The proposed algorithm is derived by extending the Takeuchi et al.'s Incremental KPCA (T-IKPCA) that can learn a new data incrementally without keeping past training data. However , even if more than two data are given in a chunk, T-IKPCA should learn them individually; that is, in order to update the eigen-feature space, the eigenvalue decomposition should be performed for every data in the chunk. To alleviate this problem, we extend T-IKPCA such that an eigen-feature space learning is conducted by performing the eigenvalue decomposition only once for a chunk of given data. In the proposed IKPCA, whenever a new chunk of training data are given, linearly independent data are first selected based on the cumulative proportion. Then, the eigenspace augmentation is conducted by calculating the coefficients for the selected linearly independent data, and the eigen-feature space is rotated based on the rotation matrix that can be obtained by solving a kernel eigenvalue problem. To verify the effectiveness of the proposed IKPCA, the learning time and the accuracy of eigenvectors are evaluated using the three VCI benchmark data sets. From the experimental results, we confirm that the proposed IKPCA can learn an eigen-feature space very fast without sacrificing the recognition accuracy.
    01/2011; DOI:10.1109/IJCNN.2011.6033599
  • [Show abstract] [Hide abstract]
    ABSTRACT: An incremental learning algorithm of Kernel Principal Component Analysis (KPCA) called Chunk Incremental KPCA (CIKPCA) has been proposed for online feature extraction in pattern recognition. CIKPCA can reduce the number of times to solve the eigenvalue problem compared with the conventional incremental KPCA when a small number of data are simultaneously given as a stream of data chunks. However, our previous work suggests that the computational costs of the independent data selection in CIKPCA could dominate over those of the eigenvalue decomposition when a large chunk of data are given. To verify this, we investigate the influence of the chunk size to the learning time in CIKPCA. As a result, CIKPCA requires more learning time than IKPCA unless a large chunk of data are divided into small chunks (e.g., less than 50).
  • [Show abstract] [Hide abstract]
    ABSTRACT: In this paper, a new process monitoring approach is proposed for multimode time-varying processes. The Kronecker product is introduced to modify the monitoring matrices. Then the original space can be separated into two different parts, which are the common part and the specific part. There are both time-varying similarity and dissimilarity in the underlying correlations of different modes, which play different roles in the industrial processes. Because the industrial processes have the non-Gaussian and nonlinear characteristics, the kernel independent component analysis (KICA) is modified to monitor the multimode time-varying processes in this paper. The global multimode basis vector and the multimode sub-basis vector are obtained based on the modified KICA. Then, the common part and specific part in one mode are, respectively, analyzed. The proposed method is applied to monitor the continuous annealing process. The proposed approach effectively extracts the non-Gaussian and nonlinear features in the different time-varying modes.
    Chemical Engineering Science 01/2013; 88:23–32. DOI:10.1016/j.ces.2012.11.008 · 2.61 Impact Factor