K-Local Hyperplane and Convex Distance Nearest Neighbor Algorithms

8 years 7 months ago
K-Local Hyperplane and Convex Distance Nearest Neighbor Algorithms
Guided by an initial idea of building a complex (non linear) decision surface with maximal local margin in input space, we give a possible geometrical intuition as to why K-Nearest Neighbor (KNN) algorithms often perform more poorly than SVMs on classification tasks. We then propose modified K-Nearest Neighbor algorithms to overcome the perceived problem. The approach is similar in spirit to Tangent Distance, but with invariances inferred from the local neighborhood rather than prior knowledge. Experimental results on real world classification tasks suggest that the modified KNN algorithms often give a dramatic improvement over standard KNN and perform as well or better than SVMs. 1 Motivation The notion of margin for classification tasks has been largely popularized by the success of the Support Vector Machine (SVM) [2, 15] approach. The margin of SVMs has a nice geometric interpretation1 : it can be defined informally as (twice) the smallest Euclidean distance between the decision s...
Pascal Vincent, Yoshua Bengio
Added 31 Oct 2010
Updated 31 Oct 2010
Type Conference
Year 2001
Where NIPS
Authors Pascal Vincent, Yoshua Bengio
Comments (0)