Annealed Discriminant Analysis

Abstract. Motivated by the analogies to statistical physics, the deterministic annealing (DA) method has successfully been demonstrated in a variety of application. In this paper, we explore a new methodology to devise the classifier under the DA method. The differential cost function is derived subject to a constraint on the randomness of the solution, which is governed by the temperature T. While gradually lowering the temperature, we can always find a good solution which can both solve the overfitting problem and avoid poor local optima. Our approach is called annealed discriminant analysis (ADA). It is a general approach, where we elaborate two classifiers, i.e., distance-based and inner product-based, in this paper. The distance-based classifier is an annealed version of linear discriminant analysis (LDA) while the inner product-based classifier is a generalization of penalized logistic regression (PLR). As such, ADA provides new insights into the workings of these two clas...

• Type: Conference
• Year: 2005
• Where: ECML
• Authors: Gang Wang, Zhihua Zhang, Frederick H. Lochovsky
• Keywords: Annealed Discriminant Analysis, Discriminant Analysis, ECML 2005, Inner Product-based Classifier