Direct Zero-Norm Optimization for Feature Selection

Zero-norm, defined as the number of non-zero elements in a vector, is an ideal quantity for feature selection. However, minimization of zero-norm is generally regarded as a combinatorially difficult optimization problem. In contrast to previous methods that usually optimize a surrogate of zero-norm, we propose a direct optimization method to achieve zero-norm for feature selection in this paper. Based on Expectation Maximization (EM), this method boils down to solving a sequence of Quadratic Programming problems and hence can be practically optimized in polynomial time. We show that the proposed optimization technique has a nice Bayesian interpretation and converges to the true zero norm asymptotically, provided that a good starting point is given. Following the scheme of our proposed zero-norm, we even show that an arbitrary-norm based Support Vector Machine can be achieved in polynomial time. A series of experiments demonstrate that our proposed EM based zeronorm outperforms other...

• Type: Conference
• Year: 2008
• Where: ICDM
• Authors: Kaizhu Huang, Irwin King, Michael R. Lyu
• Keywords: Data Mining, Difficult Optimization Problem, Feature Selection, ICDM 2008, Polynomial Time