Sciweavers

ACL
2012

Improving Word Representations via Global Context and Multiple Word Prototypes

11 years 6 months ago
Improving Word Representations via Global Context and Multiple Word Prototypes
Unsupervised word representations are very useful in NLP tasks both as inputs to learning algorithms and as extra word features in NLP systems. However, most of these models are built with only local context and one representation per word. This is problematic because words are often polysemous and global context can also provide useful information for learning word meanings. We present a new neural network architecture which 1) learns word embeddings that better capture the semantics of words by incorporating both local and global document context, and 2) accounts for homonymy and polysemy by learning multiple embeddings per word. We introduce a new dataset with human judgments on pairs of words in sentential context, and evaluate our model on it, showing that our model outperforms competitive baselines and other neural language models. 1
Eric H. Huang, Richard Socher, Christopher D. Mann
Added 29 Sep 2012
Updated 29 Sep 2012
Type Journal
Year 2012
Where ACL
Authors Eric H. Huang, Richard Socher, Christopher D. Manning, Andrew Y. Ng
Comments (0)