Article

Support vector novelty detection in hidden space

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

In this paper, a family of support vector novelty detection (or SVND) in hidden space is presented. Firstly a hidden-space SVND (or HSVND) algorithm is proposed. The data in an input space is mapped into a hidden space by using some hidden function. In the hidden space, we can perform the SVND with linear kernel. The advantage of HSVND over SVND is that a hidden function can be an arbitrary real-valued function including Mercer kernels and their combination. In SVND algorithm, only Mercer kernels can be used. Unfortunately, the mapped data may be non-spherical distributed data. In this case, HSVND works badly. Secondly a whitening HSVND (WHSVND) is proposed. We perform whitening method on the mapped data and then the SVND with linear kernel. The whitened data is spherical approximately. We prove that WHSVND is identical to whitening SVND (WSVND) if a single and same Mercer kernel is taken as a mapping function. In fact, above algorithms cannot work well if the data locates a subspace of input sample space or is rank deficiency. In this case, a mapping function only maps data from one subspace to another subspace and cannot make data spanning whole sample space. Thus noised and whitening HSVND (NWHSVND) is present for kernel mapping function. Before the data is mapped, some small randomly noise is put on it. Based on the noised data, the data is mapped into the hidden space. Next we whiten the mapped data, and perform the SVND with linear kernel. Experiments are performed on artificial and real-world data.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Hidden space support vector machine (HSSVM) extends the set of usable kernels [2], which only requires the hidden function to be symmetry. Researches on HSSVM include constructing sparse or ensemble algorithms [3][4], applying it new area [5]. However, HSSVM needs to solve a constrained program and requires long training time for large scale data. ...
Article
Based on the concept of relative density degrees, a density-punished support vector data description method is presented. The relative density degrees are associated with punishing misclassifications. If the relative density degree of the sample is large, it is likely to be a target sample. Thus, a large penalty should be put on its misclassification. Similarly, if the relative density degree of the sample is small, it might be a boundary or noise point so that the corresponding penalty for its misclassification should be small as well. The experimental results on UCI datasets show that the proposed method has better performance compared with support vector data description and density-induced support vector data description.
Article
A hidden space smooth support vector machine with particle swarm optimization (PSO-HSSSVM) algorithm is proposed to solve problems of long training time and computing complex in using hidden space support vector machine (HSSVM) to solve constrained convex quadratic programs. The proposed algorithm transforms the input data to a hidden space using a hidden function, and has no any restriction on the positive definity of the hidden function. The entropy function is employed to approximate the plus function of the slack vector, and a smooth differentiable unconstrained convex quadratic program is derived. The conjugate gradient (CG) algorithm is used to solve the smooth model, and the particle swarm optimization (PSO) algorithm is used to give the optimal parameters. Experiments show that the PSO-HSSSVM enlarges the usable kernels of smooth support vector machine (SSVM), and its accuracy and training time are similar to those of SSVM; The PSO-HSSSVM improves the accuracy of HSSVM by 2.14%, and the training time is only 9.5% that of HSSVM.
Conference Paper
Applying the smoothing techniques to the support vector machine in the hidden space, a smooth hidden space support vector machine (SHSSVM) is presented with some distinct mathematical features, such as the strong convexity and infinite differentiability. Beyond that, SHSSVM broadens the area of admissible kernel functions, where any real-valued symmetry function can be used as the hidden function, including the Mercer kernels and their combinations. Firstly, the input data are transformed to the hidden space by a hidden function. Secondly, the smoothing technique is utilized to derive the unconstrained smooth model. Finally, the Newton algorithm is introduced to figure out the optimal solution. The numerical experiments on benchmark data demonstrate that SHSSVM has much higher training accuracies than HSSVM and SSVM, but with much lower training time.
Article
This paper proposes a 1-norm support vector novelty detection (SVND) method and discusses its sparseness. 1-norm SVND is formulated as a linear programming problem and uses two techniques for inducing sparseness, or the 1-norm regularization and the hinge loss function. We also find two upper bounds on the sparseness of 1-norm SVND, or exact support vector (ESV) and kernel Gram matrix rank bounds. The ESV bound indicates that 1-norm SVND has a sparser representation model than SVND. The kernel Gram matrix rank bound can loosely estimate the sparseness of 1-norm SVND. Experimental results show that 1-norm SVND is feasible and effective.
Article
Full-text available
Hidden space support vector machines (HSSVMs) are presented in this paper. The input patterns are mapped into a high-dimensional hidden space by a set of hidden nonlinear functions and then the structural risk is introduced into the hidden space to construct HSSVMs. Moreover, the conditions for the nonlinear kernel function in HSSVMs are more relaxed, and even differentiability is not required. Compared with support vector machines (SVMs), HSSVMs can adopt more kinds of kernel functions because the positive definite property of the kernel function is not a necessary condition. The performance of HSSVMs for pattern recognition and regression estimation is also analyzed. Experiments on artificial and real-world domains confirm the feasibility and the validity of our algorithms.
Conference Paper
Full-text available
To conduct real-time video surveillance using low-cost commercial off-the-shelf hardware, system designers typically define the classifiers prior to the deployment of the system so that the performance of the system can be optimized for a particular mission. This implies the system is restricted to interpreting activity in the environment in terms of the original context specified. Ideally the system should allow the user to provide additional context in an incremental fashion as conditions change. Given the volumes of data produced by the system, it is impractical for the user to periodically review and label a significant fraction of the available data. We explore a strategy for designing a real-time object classification process that aids the user in identifying novel, informative examples for efficient incremental learning
Article
Full-text available
. When a classifier is used to classify objects, it is important to know if these objects resemble the train objects the classifier is trained with. Several methods to detect novel objects exist. In this paper a new method is presented which is based on the instability of the output of simple classifiers on new objects. The performances of the outlier detection methods is shown in a handwritten digit recognition problem. 1 Introduction A very important aspect of the use of neural networks is the ability to generalize. Good generalization means that the classifier gives reasonable responses on unseen data. This can only be achieved when the new data originates from the same distribution as from which the data is trained. When objects from a different data distribution are classified, the output responses of the classifier are completely unpredictable. To prevent these unpredictable responses the novelties have to be detected. Detection of novel objects can be done by estimating ...
Conference Paper
A new nonlinear principle component analysis (PCA) method, hidden space principal component analysis (HSPCA) is presented in this paper. Firstly, the data in the input space is mapped into a high hidden space by a nonlinear function whose role is similar to that of hidden neurons in Artificial Neural Networks. Then the goal of features extraction and data compression will be implemented by performing PCA on the mapped data in the hidden space. Compared with linear PCA method, our algorithm is a nonlinear PCA one essentially and can extract the data features more efficiently. While compared with kernel PCA method presented recently, the mapped samples are exactly known and the conditions satisfied by nonlinear mapping functions are more relaxed. The unique condition is symmetry for kernel function in HSPCA. Finally, experimental results on artificial and real-world data show the feasibility and validity of HSPCA.
Article
Unnatural patterns in the control charts can be associated with a specific set of assignable causes for process variation. Hence pattern recognition is very useful in identifying process problem. A common difficulty in existing control chart pattern recognition approaches is that of discrimination between different types of patterns which share similar features. This paper proposes an artificial neural network based model, which employs a pattern discrimination algorithm to recognise unnatural control chart patterns. The pattern discrimination algorithm is based on several special-purpose networks trained for specific recognition tasks. The performance of the proposed model was evaluated by simulation using two criteria: the percentage of correctly recognised patterns and the average run length (ARL). Numerical results show that the false recognition problem has been effectively addressed. In comparison with previous control chart approaches, the proposed model is capable of superior ARL performance while the type of the unnatural pattern can also be accurately identified.
Article
Single-class minimax probability machines (MPMs) offer robust novelty detection with distribution-free worst case bounds on the probability that a pattern will fall inside the normal region. However, in practice, they are too cautious in labeling patterns as outlying and so have a high false negative rate (FNR). In this paper, we propose a more aggressive version of the single-class MPM that bounds the best case probability that a pattern will fall inside the normal region. These two MPMs can then be used together to delimit the solution space. By using the hyperplane lying in the middle of this pair of MPMs, a better compromise between false positives (FPs) and false negatives (FNs), and between recall and precision can be obtained. Experiments on the real-world data sets show encouraging results.
Conference Paper
In this paper, a variant of support vector novelty detection (SVND) with dot product kernels is presented for non-spherical distributed data. Firstly we map the data in input space into a reproducing kernel Hilbert space (RKHS) by using kernel trick. Secondly we perform whitening process on the mapped data using kernel principal component analysis (KPCA). Finally, we adopt SVND method to train and test whitened data. Experiments were performed on artificial and real-world data.
Conference Paper
ARTMAP-FD extends fuzzy ARTMAP to perform familiarity discrimination. That is, the network learns to abstain from meaningless guesses on patterns not belonging to a class represented in the training set. ARTMAP-FD can also be applied in conjunction with sequential evidence accumulation. Its performance is illustrated here on simulated radar range profile data