Conference Paper

Optimizing a Class of Feature Selection Measures

Conference: NIPS 2009 Workshop on Discrete Optimization in Machine Learning: Submodularity, Sparsity & Polyhedra (DISCML)


Feature selection is an important processing step in machine learning and the de-sign of pattern-recognition systems. A major challenge consists in the selection of relevant features in cases of high-dimensional data sets. In order to tackle the computational complexity, heuristic, sequential or random search strategies are applied frequently. These methods, however, often yield only locally optimal fea-ture sets that might be globally sub-optimal. The aim of our research is to derive a new, efficient approach that ensures globally optimal feature sets. We focus on the so-called filter methods. We show that a number of feature-selection measures, e.g., the correlation-feature-selection measure, the minimal-redundancy-maximal-relevance measure and others, can be fused and generalized. We formulate the fea-ture selection problem as a polynomial-mixed 0 – 1 fractional programming prob-lem (P M 01F P). To solve it, we transform the P M 01F P problem into a mixed 0-1 linear programming (M 01LP) problem. This transformation is performed by applying an improved Chang's method of grouping additional variables. To ob-tain the globally optimal solution to the M 01LP problem, the branch-and-bound algorithm can be used. Experimental results obtained over the UCI database show that our globally optimal method outperforms other heuristic search procedures by up to 10 % of redundant or confusing features that are removed from the original data set, while keeping or yielding an even better accuracy.

Download full-text


Available from: Katrin Franke,
Show more