Feature Selection for Nonlinear Kernel Support Vector Machines

12 years 1 months ago
Feature Selection for Nonlinear Kernel Support Vector Machines
An easily implementable mixed-integer algorithm is proposed that generates a nonlinear kernel support vector machine (SVM) classifier with reduced input space features. A single parameter controls the reduction. On one publicly available dataset, the algorithm obtains 92.4% accuracy with 34.7% of the features compared to 94.1% accuracy with all features. On a synthetic dataset with 1000 features, 900 of which are irrelevant, our approach improves the accuracy of a full-feature classifier by over 30%. The proposed algorithm introduces a diagonal matrix E with ones for features present in the classifier and zeros for removed features. By alternating between optimizing the continuous variables of an ordinary nonlinear SVM and the integer variables on the diagonal of E, a decreasing sequence of objective function values is obtained. This sequence converges to a local solution minimizing the usual data fit and solution complexity while also minimizing the number of features used.
Olvi L. Mangasarian, Gang Kou
Added 19 Oct 2010
Updated 19 Oct 2010
Type Conference
Year 2007
Where ICDM
Authors Olvi L. Mangasarian, Gang Kou
Comments (0)