about
Neural Turing Machines [pdf] (arxiv.org)
75 points by neurologic on Oct 21, 2014 | hide | past | pdf | 11 comments on HN

In plain words: A neural network is hooked up to an external memory it can read and write, like a computer's, and the whole setup is trained so it learns what to store and retrieve. From just input-output examples, it figured out simple algorithms like copying, sorting, and recalling, which ordinary networks cannot do.

Abstract · Neural Turing Machines

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.

Alex Graves, Greg Wayne, Ivo Danihelka
arXiv:1410.5401 · cs.NE · submitted Oct 20, 2014 · updated Dec 10, 2014
abstract · pdf · html

add comment on HN
Also discussed: Sep 2026 (4 points, 0 comments) · Feb 2019 (1 point, 0 comments) · May 2016 (55 points, 27 comments) · Jan 2016 (2 points, 0 comments) · Oct 2014 (8 points, 1 comment) · Oct 2014 (60 points, 7 comments)

For arXiv submissions, please submit the abstract link; in this case: http://arxiv.org/abs/1410.5401

The advantage is immediate: One gets a plain website with the abstract, which allows the visitor to decide whether the PDF download is worth it. The PDF itself is only one more click away.

I personally would also leave out the v1, as that link automatically goes to the latest version in all cases.

Serious question: are there any browsers in common use that don't display PDFs inline? Chrome and Firefox both do, and their built-in PDF viewers seem very simple and streamlined and efficient.

It just seems a bit odd to question whether a 760KB PDF is "worth it" when the current top post on HN is a blog post that downloads 780KB of Javascript, and has attracted no such comments.

Anyway, the paper is very interesting. The sorting example is particularly impressive -- I almost wouldn't have believed it was possible to learn automatically.

I'm running Firefox on a fairly beefy laptop, and the integrated PDF viewer annoyed me so much that I disabled it some time last year. I suppose it's fine for simple stuff, and it's definitely better than an Adobe Reader plugin, but it just manages to be much too sluggish on too many real-world examples out there when standalone viewers work just fine and smooth - but unfortunately don't integrate well in the overall browsing experience.
I'm a serial computer and browser abuser, and on OS X, I can easily crash my browser, and sometimes even the whole OS, if I'm not careful what I load up in a new browser tab. So I'd rather have the simpler abstract page too.
The browser on my phone and tablet
I strongly prefer the direct link to the paper. Downloading and viewing a PDF is not so burdensome that this is a problem. I get the abstract at the top of the paper and if I'm no interested I can just close the tab that's showing the PDF same as closing a web page.
I strongly prefer the cite-able link to the pdf. The link points to the PDF, but the PDF does not point to the link. The link points to all versions of the paper.
As it turns out, the URL of the pdf is cite-able.
But it doesn't link to the versions of the paper.

You want to save one click of the mouse. I want to save having to reverse-engineer the parent URL.

Interesting read. Extrapolating from how the network learned to copy and sort data it points a bit to a future with software engineers not coding programs to do the job but training networks with the right data instead. That would basically end the language war. :) I hope time to silicon is a single digit year.
Wow, if they are publishing this, you can bet they are applying these techniques to much more sophisticated problems already.

This might be another preliminary signal for a future wave of white-collar job destruction.