An Efficient Incremental Kernel Principal Component Analysis for Online Feature Selection.
ABSTRACT In this paper, a feature extraction method for online classification problems is proposed by extending kernel principal component analysis (KPCA). In our previous work, we proposed an incremental KPCA algorithm which could learn a new input incrementally without keeping all the past training data. In this algorithm, eigenvectors are represented by a linear sum of linearly independent data which are selected from given training data. A serious drawback of the previous IKPCA is that many independent data are prone to be selected during learning and this causes large computation and memory costs. For this problem, we propose a novel approach to the selection of independent data; that is, they are not selected in the high-dimensional feature space but in the low-dimensional eigenspace spanned by the current eigenvectors. Using this method, the number of independent data is restricted to the number of eigenvectors. This restriction makes the learning of the modified IKPCA (M-IKPCA) very fast without loosing the approximation accuracy against true eigenvectors. To verify the effectiveness of M-IKPCA, the learning time and the accuracy of eigenspaces are evaluated using two UCI benchmark datasets. As a result, we confirm that the learning of M-IKPCA is at least 5 times faster than the previous version of IKPCA.
- SourceAvailable from: Nikola Kirilov KasabovEvolving Intelligent Systems: Methodology and Applications, 04/2010: pages 151 - 171; , ISBN: 9780470569962
- [Show abstract] [Hide abstract]
ABSTRACT: Kernel principal component analysis (KPCA) is known as a nonlinear feature extraction method. Takeuchi et al. have proposed an incremental type of KPCA (IKPCA) that can update an eigen-space incrementally for a sequence of data. However, in IKPCA, the eigenvalue decomposition should be carried out for every single data, even though a chunk of data is given at one time. To reduce the computational costs in learning chunk data, this paper proposes an extended IKPCA called Chunk IKPCA (CIKPCA) where a chunk of multiple data is learned with single eigenvalue decomposition. For a large data chunk, to reduce further computation time and memory usage, it is first divided into several smaller chunks, and only useful data are selected based on the accumulation ratio. In the proposed CIKPCA, a small set of independent data are first selected from a reduced set of data so that eigenvectors in a high-dimensional feature space can be represented as a linear combination of such independent data. Then, the eigenvectors are incrementally updated by keeping only an eigenspace model that consists of the sextuplet such as independent data, coefficients, eigenvalues, and mean information. The proposed CIKPCA can augment an eigen-feature space based on the accumulation ratio that can also be updated without keeping all the past data, and the eigen-feature space is rotated by solving an eigenvalue problem once for each data chunk. The experiment results show that the learning time of the proposed CIKPCA is greatly reduced as compared with KPCA and IKPCA without sacrificing recognition accuracy.Evolving Systems 01/2015; DOI:10.1007/s12530-015-9131-7
- [Show abstract] [Hide abstract]
ABSTRACT: In this paper, a new process monitoring approach is proposed for multimode time-varying processes. The Kronecker product is introduced to modify the monitoring matrices. Then the original space can be separated into two different parts, which are the common part and the specific part. There are both time-varying similarity and dissimilarity in the underlying correlations of different modes, which play different roles in the industrial processes. Because the industrial processes have the non-Gaussian and nonlinear characteristics, the kernel independent component analysis (KICA) is modified to monitor the multimode time-varying processes in this paper. The global multimode basis vector and the multimode sub-basis vector are obtained based on the modified KICA. Then, the common part and specific part in one mode are, respectively, analyzed. The proposed method is applied to monitor the continuous annealing process. The proposed approach effectively extracts the non-Gaussian and nonlinear features in the different time-varying modes.Chemical Engineering Science 01/2013; 88:23–32. DOI:10.1016/j.ces.2012.11.008 · 2.61 Impact Factor