Automatic Corpus-Based Thai Word Extraction with the C4.5 Learning Algorithm

"Word" is difficult to define in the languages that do not exhibit explicit word boundary, such as Thai. Traditional methods on defining words for this kind of languages have to depend on human judgement which bases on unclear criteria o1" procedures, and have several limitations. This paper proposes an algorithm for word extraction from Thai texts without borrowing a hand from word segmentation. We employ the c4.5 learning algorithm for this task. Several attributes such as string length, frequency, nmtual information and entropy are chosen for word/non-word determination. Our experiment yields high precision results about 85% in both training and test corpus.

• Type: Conference
• Year: 2000
• Where: COLING
• Authors: Virach Sornlertlamvanich, Tanapong Potipiti, Thatsanee Charoenporn
• Keywords: COLING 2000, COLING 2008, Explicit Word Boundary, Word Extraction, Word Segmentation