ML Newsnew | past | best | rss | submit about
Neural Machine Translation and Sequence-To-sequence Models: A Tutorial (arxiv.org)
110 points by tim_sw on Mar 7, 2017 | hide | past | pdf | 6 comments on HN

In plain words: A guide to neural sequence-to-sequence models, which read a whole sentence and generate a translation or other output one word at a time. It builds from basic intuition to the math, ending with a coding exercise to check understanding.

Abstract · Neural Machine Translation and Sequence-to-sequence Models: A Tutorial

This tutorial introduces a new and powerful set of techniques variously called "neural machine translation" or "neural sequence-to-sequence models". These techniques have been used in a number of tasks regarding the handling of human language, and can be a powerful tool in the toolbox of anyone who wants to model sequential data of some sort. The tutorial assumes that the reader knows the basics of math and programming, but does not assume any particular experience with neural networks or natural language processing. It attempts to explain the intuition behind the various methods covered, then delves into them with enough mathematical detail to understand them concretely, and culiminates with a suggestion for an implementation exercise, where readers can test that they understood the content in practice.

Graham Neubig
arXiv:1703.01619 · cs.CL, cs.LG, stat.ML · submitted Mar 5, 2017
abstract · pdf · html · 65 Pages

add comment on HN

ageitgey on Mar 7, 2017 [–]
This is a good paper for anyone interested in how modern Machine Translation works at the level of detail you might get from a well-written text in a college-level CS course (which is what I believe this is from). The paper starts with a background on statistical machine translation and then goes through the newer approach of sequence-to-sequence learning for translation, including word replacement and attention mechanisms. It's a good overview.

But if you are looking for a higher-level introduction that covers the same big ideas in ~10 minutes for a more general audience, here's my take: https://medium.com/@ageitgey/machine-learning-is-fun-part-5-...

reply
denzil_correa on Mar 7, 2017 | parent [–]
This is one of my favorite general high level introduction to this area. Most people whom I have shared this with understand it even if they do not have deep technical knowledge. Great material!
reply
rrherr on Mar 8, 2017 | parent [–]
Adam, I've been using Machine Learning Is Fun Part 1 at work, to introduce non-technical business leaders to supervised machine learning concepts. Thanks for the great series!
reply
ericjang on Mar 8, 2017 [–]
Stephen Merity of Metamind has a nice visual tutorial here as well: http://smerity.com/articles/2016/google_nmt_arch.html
reply
nourishingvoid on Mar 8, 2017 [–]
I was lucky enough to study in the same lab as the author while he was doing his doctorate. Graham has a real talent for explaining complicated concepts in a way that's easy to understand. He's also strongly committed to putting as much of his code and data as he can online so that anyone can play around with it, including people who aren't academics.
reply
iraphael on Mar 8, 2017 [–]
If you're interested in a more introductory talk, I gave one a few weeks ago that goes over the basics of Deep Learning, and how TensorFlow works internally.

https://www.youtube.com/watch?v=DYlHnxfrrZY

reply
About | RSS | RSS (all) | HN arXiv