logo Idiap Research Institute        
 [BibTeX] [Marc21]
Is Deep Learning Really Necessary for Word Embeddings?
Type of publication: Idiap-RR
Citation: Lebret_Idiap-RR-44-2013
Number: Idiap-RR-44-2013
Year: 2013
Month: 12
Institution: Idiap
Note: Accepted to NIPS Deep Learning Workshop
Abstract: Word embeddings resulting from neural language models have been shown to be successful for a large variety of NLP tasks. However, such architecture might be difficult to train and time-consuming. Instead, we propose to drastically sim- plify the word embeddings computation through a Hellinger PCA of the word co-occurence matrix. We compare those new word embeddings with some well- known embeddings on NER and movie review tasks and show that we can reach similar or even better performance. Although deep learning is not really necessary for generating good word embeddings, we show that it can provide an easy way to adapt embeddings to specific tasks.
Projects Idiap
Authors Lebret, Rémi
Legrand, Joël
Collobert, Ronan
Added by: [ADM]
Total mark: 0
  • Lebret_Idiap-RR-44-2013.pdf (MD5: de3cbf7a73008ee60324b40a0a7f7105)