ML Newsnew | past | best | rss | submit about
Exploiting Similarities Among Languages for Machine Translation (2013) (arxiv.org)
6 points by ColinWright on Aug 22, 2015 | hide | past | pdf | 1 comment on HN

In plain words: It learns word meaning from text in each language, then lines up the two using a small bilingual list to fill in missing entries. For English and Spanish, the right word was in its top five guesses almost 90% of the time, without hand-built dictionaries.

Abstract · Exploiting Similarities among Languages for Machine Translation

Dictionaries and phrase tables are the basis of modern statistical machine translation systems. This paper develops a method that can automate the process of generating and extending dictionaries and phrase tables. Our method can translate missing word and phrase entries by learning language structures based on large monolingual data and mapping between languages from small bilingual data. It uses distributed representation of words and learns a linear mapping between vector spaces of languages. Despite its simplicity, our method is surprisingly effective: we can achieve almost 90% precision@5 for translation of words between English and Spanish. This method makes little assumption about the languages, so it can be used to extend and refine dictionaries and translation tables for any language pairs.

Tomas Mikolov, Quoc V. Le, Ilya Sutskever
arXiv:1309.4168 · cs.CL · submitted Sep 17, 2013
abstract · pdf · html

add comment on HN

deepnet on Aug 22, 2015 [–]
This work indicates that language translation is vector addition on a lower dimensional semantic manifold that neural networks converge on, derived from the high dimensional whole vocabulary context vectors of the input.

It is extended by :

Sequence to Sequence Learning with Neural Networks by Sutskever, Vinyals & Le

http://arxiv.org/abs/1409.3215

which adds LTSM, a trainable deep neural net type of long term memory - which allows whole sentences to be digested and translated.

" Our main result is that on an English to French translation task from the WMT'14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8 on the entire test set, ... Additionally, the LSTM did not have difficulty on long sentences."[1]

another interesting result is the translation of pixels to words that describe the image.

Deep Fragment Embeddings for Bidirectional Image Sentence Mapping by Karpathy, Joulin, & Li

http://papers.nips.cc/paper/5281-deep-fragment-embeddings-fo...

code & a demo is available :

http://cs.stanford.edu/people/karpathy/deepimagesent/

reply
About | RSS | RSS (all) | HN arXiv