about
Weakly supervised memory networks [pdf] (arxiv.org)
1 point by dave_sullivan on Apr 1, 2015 | hide | past | pdf | discuss on HN

In plain words: A network that reads from a large stored memory several times before replying, learning the whole process at once so it needs fewer hand-labeled hints. It matched earlier memory models on question answering and usual word-prediction models on text, with extra reads helping most.

Abstract · End-To-End Memory Networks

We introduce a neural network with a recurrent attention model over a possibly large external memory. The architecture is a form of Memory Network (Weston et al., 2015) but unlike the model in that work, it is trained end-to-end, and hence requires significantly less supervision during training, making it more generally applicable in realistic settings. It can also be seen as an extension of RNNsearch to the case where multiple computational steps (hops) are performed per output symbol. The flexibility of the model allows us to apply it to tasks as diverse as (synthetic) question answering and to language modeling. For the former our approach is competitive with Memory Networks, but with less supervision. For the latter, on the Penn TreeBank and Text8 datasets our approach demonstrates comparable performance to RNNs and LSTMs. In both cases we show that the key concept of multiple computational hops yields improved results.

Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus
arXiv:1503.08895 · cs.NE, cs.CL · submitted Mar 31, 2015 · updated Nov 24, 2015
abstract · pdf · html · Accepted to NIPS 2015

add comment on HN