about
A Brief Survey of Deep Reinforcement Learning (arxiv.org)
174 points by tim_sw on Aug 22, 2017 | hide | past | pdf | 3 comments on HN

In plain words: This survey explains how deep neural networks let agents learn from camera images, covering two main families: scoring actions and tuning the action chooser directly. This combination handles tasks once out of reach, like playing video games from pixels and steering robots from cameras.

Abstract

Deep reinforcement learning is poised to revolutionise the field of AI and represents a step towards building autonomous systems with a higher level understanding of the visual world. Currently, deep learning is enabling reinforcement learning to scale to problems that were previously intractable, such as learning to play video games directly from pixels. Deep reinforcement learning algorithms are also applied to robotics, allowing control policies for robots to be learned directly from camera inputs in the real world. In this survey, we begin with an introduction to the general field of reinforcement learning, then progress to the main streams of value-based and policy-based methods. Our survey will cover central algorithms in deep reinforcement learning, including the deep $Q$-network, trust region policy optimisation, and asynchronous advantage actor-critic. In parallel, we highlight the unique advantages of deep neural networks, focusing on visual understanding via reinforcement learning. To conclude, we describe several current areas of research within the field.

Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, Anil Anthony Bharath
arXiv:1708.05866 · cs.LG, cs.AI, cs.CV, stat.ML · submitted Aug 19, 2017 · updated Sep 28, 2017
abstract · pdf · html · IEEE Signal Processing Magazine, Special Issue on Deep Learning for Image Understanding (arXiv extended version)

add comment on HN

Another recent review paper: Deep Reinforcement Learning: An Overview, https://arxiv.org/abs/1701.07274
This was a horribly written paper. It includes all the popular buzzwords but the authors ability to convey the meaning of these concepts was utterly lacking. Don't expect to learn anything at all from this one. OP's article is far better than this.
I am the author of Deep Reinforcement Learning: An Overview. I fully understand my overview is far from perfect, and I warmly welcome (constructive and responsible) comments and criticisms.

My overview was published on arXiv on Jan 25, 2017, with major updates on July 15, 2017, much earlier than the brief survey of DRL, submitted to arXiv on Aug 19 2017 (manuscript to the special issue due on March 1, 2017).

The brief survey did not cite my overview. There were some discussions about it on Twitter a couple days ago when the brief survey appeared online. I thought it was friendly handled. I realized that it was not, when I saw this comment.

As a result, 1) I registered an account on YCombinator, with my real name, and reply now; 2) in the meanwhile, I sent a formal request to the authors and the editors of the brief survey to cite my overview, and make comments on it, positively or negatively.