Another nice companion survey to go with this excellent work:

Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art

https://arxiv.org/abs/1704.05519

140 papers cited, smashing :)

Rodney Brooks famously condemned simulation and games positing that AI can only develop in an embodied agents.

Robotics with deepnets is the edge and Berkeley's Abeel, Levine and Finn are right on it (IMHO). This paper puts it all in valuable context.

Anyone else going to the DeepRL Bootcamp at Berkeley in August ?

> Rodney Brooks famously condemned simulation and games positing that AI can only develop in an embodied agents.

This is a religious view and preposterous. It's a milder form of the (surprisingly common) belief that only humans can be intelligent.

That claim is false even if embodied cognition is true (from Wikipedia):

"Embodied cognition is the theory that many features of cognition, whether human or otherwise, are shaped by aspects of the entire body of the organism"

That was Brooks' reaction against the logic-based AI of the 1980s. Around 1990 I went to a talk where Brooks was plugging the Cog project he was starting.[1] Brooks had some success with purely reactive insect-level AI, and was trying to make the big jump to human-level AI. I asked him, why not try for mouse-level AI. That might be within reach. He said "Because I don't want to go down in history as the man who developed the world's greatest robot mouse."

This is a classic problem with AI researchers. Somebody gets a good result, and then they start thinking strong human-level AI is right around the corner. AI went through this with search, planning, the General Problem Solver, perceptrons, the first generation of neural networks, and expert systems. Then came the "AI winter", late 1980s to early 2000s, when almost all the AI startups went bust. We're seeing some of it again in the machine learning / deep neural net era.

This time looks more promising, partly because we can throw more compute power at the problem. Many of the ideas in machine learning and neural nets are old, and are so inefficient that they were hopeless until people could beat on them with racks of GPUs.

The big difference this time is that AI is profitable. The field used to be tiny - maybe 20-30 people at MIT, Stanford, and CMU, with a few small groups elsewhere. Now there are hundreds of thousands of researchers, and profitable applications.

(I went through Stanford just as the expert system boom was collapsing and the "AI winter" was beginning. I met most of the big names from the logic-based AI era. It was kind of sad.)

[1] https://en.wikipedia.org/wiki/Cog_(project)

The hundreds of thousands of researchers number, is that just a guess? If it isn't I'd love to see the source.

This research (https://blog.openai.com/robots-that-learn/) is extremely relevant to Rodney Brooks' condemnation and an amazing achievement in my opinion.

That is quite some condemnation :) I am certainly interested and hope you will flesh out your positions.

Certainly Brooks ran into Moravecs paradox, that walking is the hardest form of thinking and playing chess and planning less so. And so Brooksian embodied robotics never got to the symbolic level.

Contrary to your anthropocentric claim, Brooks (and followers such as Tilden) were biomimetically inspired by the intelligence of insect locomotion and 'intelligent' engineering like the underslung chassy of spiders and beetle.

I do agree with Brooks that AI will not emerge in simulation due to the lack of real stochasticity and resolution.

[/edit] I will temper my stridency and say that it is less likely, my initial absolutist position was merely excitement and really simulation has a big role to play as imagination and plan testing. We live in a world of perception and as [Richard] Gregory says "perception is only a hypothesis, based on our prior knowledge and updated by reality".

For instance the Atari benchmarks appears random but the randomness is Mersenne Twisters with few seeds and most expert (human) arcade players learn the patterns.

That is not to say that Mnih wt al.s Nature paper wasn't groundbreaking and their Deep-Q RL is certainly insightful, promising and useful but (IMHO) the AI 'stack' and LeCun's Predictive Learning will not come about outside of a real robot acting in the world and using raw sensor data to power actuators directly.

My position is not entirely Brooksian. I follow Esther Thelen in the view that a being ( a brain and a body ) needs to learn to be by being and moving and feeling and touching.

I do apologise if my brevity and excitement appears rude, AI could well emerge in computers running simulations and in fact I think that simulation can play the role of planning and imagination but it will need to be grounded in a real physical agent - a hybrid approach will bear the most fruit.

A friend of mine awoke from a coma a month before he could move or speak. He could hear the nurses, they thought he was a vegetable.

Are you saying the nurses were right? Without the ability to move, he wasn't actually intelligent?

Edit: I see your edit now, sure I like your restatement "simulation has a big role to play as imagination and plan testing"

You edited your comment after I replied. I am sorry about your friend, I hope they are better.

I am not saying that at all. Daniel Dennet is better placed to speculate on matters like that.

[/edit] perhaps there is some common ground. Brooks' Behaviour-Based position, though useful at the time, threw out the symbolic AI with the bathwater of computational complexity that robotics was quagmired in then.

> Anyone else going to the DeepRL Bootcamp at Berkeley in August ?

I was really looking forward to it, especially because among other things, I wanted to discuss the very arguments of the absence of true randomness in simulations that you mentioned below in context of RL -- it seems there hasn't been a lot of work in analyzing the situation with simulations lately because everyone tends to get excited upon seeing the algorithm figure out bipedal movement in a variety of simulated environments et cetera. Out of sheer bad luck, I have something else scheduled around the same dates that I cannot skip :(. Oh well.

I'm really looking forward to the Berkeley bootcamp! I wonder if there is any online forum or discussion group for participants, or maybe we should form one? It would be great to get introduced to folks in advance, especially because the workshop is very short.

One of the things that really excites me is the possibility of conceptual breakthroughs, the following paper as an example direction: https://arxiv.org/abs/1611.03852

Great paper thanks!

Per your useful suggestion I made a subreddit as an online forum/ discussion group for participants ( in the 2017 Berkeley DeepRLBootcamp ).

https://deeprlbootcamp.reddit.com/

Simulation has gotten enormously better since he made that argument (late 80's, I think). You couldn't even render shaded 3D graphics in real time.

Also, while you can learn pretty good driving in simulation, you may not be able to achieve AGI. AGI requires learning to interact with people, which you can't simulate because you don't have AGI yet.

Yes I'll be at the DeepRL Bootcamp, looking forward to it. Big fan of the work coming out of Abbeel's lab and Berkeley.

I'm on the ML team at Dropbox; recently finished working through David Silver's Deep RL course on my side time. Tell me more about yourself and your interest in Deep RL!

Hi Brad,

Great, I am flying over from Scotland for the Berkeley DeepRL bootcamp.

I too am a big fan of Abeel's lab. His papers with Levine and Finn - very far sighted stuff. I have been working through the CS294-112 Deep RL course materials [1] recently.

A bio might be a touch off topic for this thread, very happy to enthuse about Neural Nets, Reinforcement Learning and Robots though - I can be emailed at deepnet at jmhz dot net

[1] https://www.youtube.com/watch?v=f4gKhK8Q6mY&list=PLkFD6_40KJ...

The above links to Pieter Abeel's CS294-112 lecture on Adversarial Nets and his Thesis on Helicopter Contollers with Andrew Ng.

Interesting stuff! I need to free up a weekend for this :)