Conference Paper

An Efficient Incremental Kernel Principal Component Analysis for Online Feature Selection

Kobe Univ., Kobe
DOI: 10.1109/IJCNN.2007.4371325 Conference: Proceedings of the International Joint Conference on Neural Networks, IJCNN 2007, Celebrating 20 years of neural networks, Orlando, Florida, USA, August 12-17, 2007
Source: DBLP


In this paper, a feature extraction method for online classification problems is proposed by extending kernel principal component analysis (KPCA). In our previous work, we proposed an incremental KPCA algorithm which could learn a new input incrementally without keeping all the past training data. In this algorithm, eigenvectors are represented by a linear sum of linearly independent data which are selected from given training data. A serious drawback of the previous IKPCA is that many independent data are prone to be selected during learning and this causes large computation and memory costs. For this problem, we propose a novel approach to the selection of independent data; that is, they are not selected in the high-dimensional feature space but in the low-dimensional eigenspace spanned by the current eigenvectors. Using this method, the number of independent data is restricted to the number of eigenvectors. This restriction makes the learning of the modified IKPCA (M-IKPCA) very fast without loosing the approximation accuracy against true eigenvectors. To verify the effectiveness of M-IKPCA, the learning time and the accuracy of eigenspaces are evaluated using two UCI benchmark datasets. As a result, we confirm that the learning of M-IKPCA is at least 5 times faster than the previous version of IKPCA.

13 Reads
  • Source
    • "Finally, PCA and LDA transform inputs into linear features, and these features are not always effective for classification purposes. Recently kernel PCA and kernel LDA are widely noticed as high-performance feature extraction methods (Scholkopf 1996); hence, the extension of incremental learning approach to kernel PCA (Takeuchi 2007 "
    [Show abstract] [Hide abstract]
    ABSTRACT: This chapter introduces an adaptive method for evolving intelligent systems in which Incremental Principal Component Analysis (IPCA) and Evolving Clustering Method (ECM) are effectively combined. This learning scheme gives a new concept for pattern recognition systems: Feature extraction and classifier learning are simultaneously carried out online. One drawback of this approach was scalability in terms of the number of data samples and number of their attributes. To overcome this problem, the authors developed a new method called Chunk IPCA (CIPCA), in which a chunk of training samples can be applied at one time to update an eigenspace model incrementally. In addition, a nearest-neighbor classifier whose prototypes are adapted to the evolution of a feature space was proposed. The chapter explains the algorithm of eigen-axis selection, the introduction of accumulation ratio as a criterion of eigen-axis augmentation, and the integration of two types of incremental learning for feature space and classifier. learning systems; principal component analysis; workstation clusters
    Full-text · Chapter · Apr 2010
  • [Show abstract] [Hide abstract]
    ABSTRACT: Least squares support vector machine (LSSVM) has been used in soft sensor modeling in recent years. In developing a successful model based on LSSVM, the first important step is feature extraction. Principal components analysis (PCA) is a usual method for linear feature extraction and kernel PCA (KPCA) is a nonlinear PCA developed by using the kernel method. KPCA can efficiently extract the nonlinear relationship between original inputs. This paper proposes to combine KPCA with LSSVM to forecast the Mooney-viscosity of styrene butadiene rubber (SBR). KPCA is firstly applied for feature extraction. Then LSSVM is applied to proceed regression modeling. The experiment results show that KPCA-LSSVM features high learning speed, good approximation and generalization ability compared with SVM and PCA-SVM. The root mean square errors of the Mooney-viscosity in the KPCA-LSSVM, PCA-LSSVM and LSSVM are 0.0145, 0.0377 and 0.1775 respectively. LSSVM with KPCA for feature extraction has best performance. It may be used to efficiently guide production.
    No preview · Article · Sep 2008
  • [Show abstract] [Hide abstract]
    ABSTRACT: Hydrogen gas concentration forecasting and evaluation is very important for Bio-ethanol Steam Reforming hydrogen production. A lot of methods have been applied in the field of gas concentration forecasting including principal component analysis (PCA) and artificial neural network (ANN) etc. this paper used kernel principal component analysis (KPCA) as a preprocessor of Least Squares Support Vector Machine (LS-SVM) to extract the principal features of original data and employed the Particle Swarm Optimization (PSO) to optimize the free parameters of LS-SVM. Then LS-SVM is applied to proceed hydrogen gas concentration regression modeling. The experiment results show that KPCA-LSSVM features high learning speed, good approximation and generalization ability compared with SVM and PCA-SVM.
    No preview · Chapter · Dec 2009
Show more