Feature Engineering for Text Classification

Most research in text classification to date has used a “bag of words” representation in which each feature corresponds to a single word. This paper examines some alternative ways to represent text based on syntactic and semantic relationships between words (phrases, synonyms and hypernyms). We describe the new representations and try to justify our hypothesis that they could improve the performance of a rule-based learner. The representations are evaluated using the RIPPER learning algorithm on the Reuters-21578 and DigiTrad test corpora. On their own the new representations are not found to produce significant performance improvements. We also try combining classifiers based on different representations using a majority voting technique, and this improves performance on both test collections. In our opinion, more sophisticated Natural Language Processing techniques need to be developed before better text representations can be produced for classification.

• Type: Conference
• Year: 1999
• Where: ICML
• Authors: Sam Scott, Stan Matwin
• Keywords: Better Text Representations, DigiTrad Test Corpora, ICML 1999, Machine Learning, RIPPER Learning Algorithm