Title: Improving Word Representations via Global Context and Multiple Word Prototypes
Abstract: Unsupervised word representations are very useful in NLP tasks both as inputs to learning algorithms and as extra word features in NLP systems. However, most of these models are built with only local context and one representation per word. This is problematic because words are often polysemous and global context can also provide useful information for learning word meanings. We present a new neural network architecture which 1) learns word embeddings that better capture the semantics of words by incorporating both local and global document context, and 2) accounts for homonymy and polysemy by learning multiple embeddings per word. We introduce a new dataset with human judgments on pairs of words in sentential context, and evaluate our model on it, showing that our model outperforms competitive baselines and other neural language models.
Publication Year: 2012
Publication Date: 2012-07-08
Language: en
Type: article
Access and Citation
Cited By Count: 1111
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot