The impact of sample reduction on PCA-based feature extraction for supervised learning

“The curse of dimensionality” is pertinent to many learning algorithms, and it denotes the drastic raise of computational complexity and classification error in high dimensions. In this paper, different feature extraction (FE) techniques are analyzed as means of dimensionality reduction, and constructive induction with respect to the performance of Naïve Bayes classifier. When a data set contains a large number of instances, some sampling approach is applied to address the computational complexity of FE and classification processes. The main goal of this paper is to show the impact of sample reduction on the process of FE for supervised learning. In our study we analyzed the conventional PCA and two eigenvector-based approaches that take into account class information. The first class-conditional approach is parametric and optimizes the ratio of between-class variance to the within-class variance of the transformed data. The second approach is a nonparametric modification of the ...

• Type: Conference
• Year: 2006
• Where: SAC
• Authors: Mykola Pechenizkiy, Seppo Puuronen, Alexey Tsymbal
• Keywords: Applied Computing, Computational Complexity, Feature Extraction, SAC 2006, Supervised Learning