Comparing Dimension Reduction Techniques for Document Clustering

In this research, a systematic study is conducted of four dimension reduction techniques for the text clustering problem, using five benchmark data sets. Of the four methods -- Independent Component Analysis (ICA), Latent Semantic Indexing (LSI), Document Frequency (DF) and Random Projection (RP) -- ICA and LSI are clearly superior when the k-means clustering algorithm is applied, irrespective of the data sets. Random projection consistently returns the worst results, where this appears to be due to the noise distribution characterizing the document clustering task.

• Type: Conference
• Year: 2005
• Where: AI
• Authors: Bin Tang, Michael A. Shepherd, Malcolm I. Heywood, Xiao Luo
• Keywords: AI 2005, Artificial Intelligence, Data Sets, Random Projection, Text Clustering Problem