Creating diverse nearest-neighbour ensembles using simultaneous metaheuristic feature selection

The nearest-neighbour (1NN) classifier has long been used in pattern recognition, exploratory data analysis, and data mining problems. A vital consideration in obtaining good results with this technique is the choice of distance function, and correspondingly which features to consider when computing distances between samples. In recent years there has been an increasing interest in creating ensembles of classifiers in order to improve classification accuracy. This paper proposes a new ensemble technique which combines multiple 1NN classifiers, each using a different distance function, and potentially a different set of features (feature vector). These feature vectors are determined for each distance metric simultaneously using Tabu Search to minimise the ensemble error rate. We show that this approach implicitly selects for a diverse set of classifiers, and by doing so achieves greater performance improvements than can be achieved by treating the classifies independently, or u...

• Type: Journal
• Year: 2010
• Where: PRL
• Authors: Muhammad Atif Tahir, Jim E. Smith
• Keywords: Different Distance, Ensemble Classifiers, PRL 2010, Tabu Search