Adaptive Term Weighting through Stochastic Optimization

12 years 3 months ago
Adaptive Term Weighting through Stochastic Optimization
Term weighting strongly influences the performance of text mining and information retrieval approaches. Usually term weights are determined through statistical estimates based on static weighting schemes. Such static approaches lack the capability to generalize to different domains and different data sets. In this paper, we introduce an on-line learning method for adapting term weights in a supervised manner. Via stochastic optimization we determine a linear transformation of the term space to approximate expected similarity values among documents. We evaluate our approach on 18 standard text data sets and show that the performance improvement of a k-NN classifier ranges between 1% and 12% by using adaptive term weighting as preprocessing step. Further, we provide empirical evidence that our approach is efficient to cope with larger problems.
Michael Granitzer
Added 02 Sep 2010
Updated 02 Sep 2010
Type Conference
Year 2010
Authors Michael Granitzer
Comments (0)