An Experiment Stemming Non-Traditional Text

Stemming is a technique which aims to extract common suffixes of words. Thus, words which are literally differhave a common stem, may be abstracted by their common stem. The underlying goal when using a stemming technique is to improve recall, at the possible expense of precision loss. A well known technique for stemming text is Porter's algorithm, which is based on a set of rules extracted from the English language. In this paper, we argue that such an algorithm it is not efficient for non-traditional texts, e.g., one made up mainly of medical terms. We thus investigate the use of a technique, called Peak-and-Plateau, which is based on tries, and compare it to Porter's algorithm. Our experiments have shown that using Porter's algorithm or none at all makes no difference as far as precision and recall goes. On the other hand, using the Peakand-Plateau technique we improved recall by about 15% and decreased precision by an average of 40%. Moreover, it compressed the orig...

• Type: Conference
• Year: 1998
• Where: SPIRE
• Authors: Mario A. Nascimento, Adriano C. R. da Cunha
• Keywords: Common Stem, Common Suffixes, Information Management, Possible Expense, SPIRE 1998