REPORT Lebret_Idiap-RR-29-2013/IDIAP Word Embeddings through Hellinger PCA Lebret, Rémi Collobert, Ronan EXTERNAL https://publications.idiap.ch/attachments/reports/2013/Lebret_Idiap-RR-29-2013.pdf PUBLIC Idiap-RR-29-2013 2013 Idiap August 2013 Word embeddings resulting from neural lan- guage models have been shown to be successful for a large variety of NLP tasks. However, such architecture might be difficult to train and time-consuming. Instead, we propose to drastically simplify the word embeddings computation through a Hellinger PCA of the word co-occurence matrix. We compare those new word embeddings with the Collobert and Weston (2008) embeddings on several NLP tasks and show that we can reach similar or even better performance.