Generalized discriminant analysis: a matrix exponential approach.

Department of Computer Science, Chongqing University, Chongqing 400030, China.
IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics: a publication of the IEEE Systems, Man, and Cybernetics Society (Impact Factor: 3.78). 08/2009; 40(1):186-97. DOI: 10.1109/TSMCB.2009.2024759
Source: PubMed

ABSTRACT Linear discriminant analysis (LDA) is well known as a powerful tool for discriminant analysis. In the case of a small training data set, however, it cannot directly be applied to high-dimensional data. This case is the so-called small-sample-size or undersampled problem. In this paper, we propose an exponential discriminant analysis (EDA) technique to overcome the undersampled problem. The advantages of EDA are that, compared with principal component analysis (PCA) + LDA, the EDA method can extract the most discriminant information that was contained in the null space of a within-class scatter matrix, and compared with another LDA extension, i.e., null-space LDA (NLDA), the discriminant information that was contained in the non-null space of the within-class scatter matrix is not discarded. Furthermore, EDA is equivalent to transforming original data into a new space by distance diffusion mapping, and then, LDA is applied in such a new space. As a result of diffusion mapping, the margin between different classes is enlarged, which is helpful in improving classification accuracy. Comparisons of experimental results on different data sets are given with respect to existing LDA extensions, including PCA + LDA, LDA via generalized singular value decomposition, regularized LDA, NLDA, and LDA via QR decomposition, which demonstrate the effectiveness of the proposed EDA method.

1 Follower
  • [Show abstract] [Hide abstract]
    ABSTRACT: Recently, various clustering approaches were proposed that incorporated both local and global information in an image dataset to learn nonlinear manifold. However, we have to calibrate a number of clustering parameters in these clustering models. In this study, we propose novel less-parameterized Exponential Discriminative Regularization with Nonnegative constraint (NESDR) clustering model based on exponential discriminant analysis in which the matrix singularity problem of discriminant analysis is handled by projecting scatter matrices in exponential domain. Further, we observed that well separated images at local level cannot be achieved using pixel value based image features for challenging image databases that contain images with different expressions, pose, illumination, or occlusion variations. We used Gist image features with proposed NEDSR model. We compared clustering performance of NEDSR-Gist with state-of-the-art existing clustering models such as Kmeans, discriminative Kmeans, normalized cut, local discriminant model and global integration, and NSDR. We report significant overall performance improvement of 8.9% (clustering accuracy) and 10.8% (normalized mutual information) on 6 benchmark image datasets over near competitor NSDR model.
    2013 International Conference on Emerging Technologies (ICET); 12/2013
  • [Show abstract] [Hide abstract]
    ABSTRACT: Linear discriminant analysis (LDA) is a powerful dimensionality reduction technique, which has been widely used in many applications. Although, LDA is well-known for its discriminant capability, it clearly does not capture the geometric structure of the data. However, from the geometric perspective, the high-dimensional data resides on some low-dimensional manifolds in the sample space and these manifold structures are essential for data clustering and classification. In this paper, we propose a novel LDA algorithm named graph regularized linear discriminant analysis (GRLDA) to further improve the conventional LDA by incorporating such geometric information of data. GRLDA is achieved by penalizing the LDA with a Graph regularization, which is an affinity matrix encoding the geometric relationship of the data points. To take high-order geometric relationship among samples into consideration, we generalize GRLDA via using the hypergraph regularization instead of the graph regularization. We name this new version as hyper graph regularized linear discriminant analysis. Moreover, we exploit the null space of LDA via using an identity matrix to regularize the between-class scatter matrix. This strategy can further improve the discriminating power of LDA algorithms. Four popular face databases are used to evaluate our proposed LDA algorithms and the results of experiments demonstrate that they outperform the state-of-the-art dimensionality reduction algorithms.
    Formal Pattern Analysis & Applications 01/2015; DOI:10.1007/s10044-014-0434-2 · 0.74 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: With the development of artificial intelligence and machine vision, face recognition has become a hot topic of pattern recognition.This paper describes the review-based comparison and recognition of challenges using holistic and hybrid appearance based approaches and the recent techniques used for improving recognition accuracy. The accuracy or efficiency of the techniques depends on the situation where the system is used. In addition, several major issues for further research in the area of face recognition are also pointed out for further improvement.