An Efficient Incremental Kernel Principal Component Analysis for Online Feature Selection.
ABSTRACT In this paper, a feature extraction method for online classification problems is proposed by extending kernel principal component analysis (KPCA). In our previous work, we proposed an incremental KPCA algorithm which could learn a new input incrementally without keeping all the past training data. In this algorithm, eigenvectors are represented by a linear sum of linearly independent data which are selected from given training data. A serious drawback of the previous IKPCA is that many independent data are prone to be selected during learning and this causes large computation and memory costs. For this problem, we propose a novel approach to the selection of independent data; that is, they are not selected in the high-dimensional feature space but in the low-dimensional eigenspace spanned by the current eigenvectors. Using this method, the number of independent data is restricted to the number of eigenvectors. This restriction makes the learning of the modified IKPCA (M-IKPCA) very fast without loosing the approximation accuracy against true eigenvectors. To verify the effectiveness of M-IKPCA, the learning time and the accuracy of eigenspaces are evaluated using two UCI benchmark datasets. As a result, we confirm that the learning of M-IKPCA is at least 5 times faster than the previous version of IKPCA.
- SourceAvailable from: Ming-Hsuan Yang
Conference Proceeding: Object tracking using incremental Fisher discriminant analysis[show abstract] [hide abstract]
ABSTRACT: This work presents a novel object tracking algorithm using incremental Fisher linear discriminant (FLD) algorithm. The sample distribution of the target class is modeled by a single Gaussian and the non-target background class is modeled by a mixture of Gaussians. To a facilitate a multiclass classification problem, we recast the classic FLD algorithm in which the number of classes does not need to be pre-determined. The most discriminant projection matrix that best separates the samples in the projected space is computed using FLD at each frame. Based on the current target location, an efficient sampling algorithm is used to predict the possible locations in the next frame. Using the current projection matrix computed by FLD, the most likely candidate which is closed to the center of the target class in the projected space is selected. Since the FLD is repeatedly computed at each frame, we develop an incremental and efficient method to compute the projection matrix based on the previous results. Experimental results show that our tracker is able to follow the target with large lighting, pose and expression variation.Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on; 09/2004
Conference Proceeding: A Novel Scalable Algorithm for Supervised Subspace Learning[show abstract] [hide abstract]
ABSTRACT: Subspace learning approaches aim to discover important statistical distribution on lower dimensions for high dimensional data. Methods such as principal component analysis (PCA) do not make use of the class information, and linear discriminant analysis (LDA) could not be performed efficiently in a scalable way. In this paper, we propose a novel highly scalable supervised subspace learning algorithm called as supervised Kampong measure (SKM). It assigns data points as close as possible to their corresponding class mean, simultaneously assigns data points to be as far as possible from the other class means in the transformed lower dimensional subspace. Theoretical derivation shows that our algorithm is not limited by the number of classes or the singularity problem faced by LDA. Furthermore, our algorithm can be executed in an incremental manner in which learning is done in an online fashion as data streams are received. Experimental results on several datasets, including a very large text data set RCV1, show the outstanding performance of our proposed algorithm on classification problems as compared to PCA, LDA and a popular feature selection approach, information gain (IG).Data Mining, 2006. ICDM '06. Sixth International Conference on; 01/2007
- [show abstract] [hide abstract]
ABSTRACT: Appearance-based image analysis techniques require fast computation of principal components of high-dimensional image vectors. We introduce a fast incremental principal component analysis (IPCA) algorithm, called candid covariance-free IPCA (CCIPCA), used to compute the principal components of a sequence of samples incrementally without estimating the covariance matrix (so covariance-free). The new method is motivated by the concept of statistical efficiency (the estimate has the smallest variance given the observed data). To do this, it keeps the scale of observations and computes the mean of observations incrementally, which is an efficient estimate for some well known distributions (e.g., Gaussian), although the highest possible efficiency is not guaranteed in our case because of unknown sample distribution. The method is for real-time applications and, thus, it does not allow iterations. It converges very fast for high-dimensional image vectors. Some links between IPCA and the development of the cerebral cortex are also discussed.IEEE Transactions on Pattern Analysis and Machine Intelligence 09/2003; 25(8):1034- 1040. · 4.80 Impact Factor