In plain words: Two simpler neural designs learn a list of numbers for each word from huge amounts of text, capturing what words mean and how they relate. They beat earlier neural network approaches on word similarity tests, learning from 1.6 billion words in under a day.
Abstract · Efficient Estimation of Word Representations in Vector Space
We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.
Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean
arXiv:1301.3781 · cs.CL · submitted Jan 16, 2013 · updated Sep 7, 2013
abstract · pdf · html