On the Role of Lexical Features in Sequence Labeling

10 years 11 months ago
On the Role of Lexical Features in Sequence Labeling
We use the technique of SVM anchoring to demonstrate that lexical features extracted from a training corpus are not necessary to obtain state of the art results on tasks such as Named Entity Recognition and Chunking. While standard models require as many as 100K distinct features, we derive models with as little as 1K features that perform as well or better on different domains. These robust reduced models indicate that the way rare lexical features contribute to classification in NLP is not fully understood. Contrastive error analysis (with and without lexical features) indicates that lexical features do contribute to resolving some semantic and complex syntactic ambiguities
Yoav Goldberg, Michael Elhadad
Added 17 Feb 2011
Updated 17 Feb 2011
Type Journal
Year 2009
Authors Yoav Goldberg, Michael Elhadad
Comments (0)