about
PathNet: Evolution Channels Gradient Descent in Super Neural Networks (arxiv.org)
60 points by jweissman on Feb 18, 2017 | hide | past | pdf | 6 comments on HN

In plain words: Small routes through one giant neural network compete and evolve to pick which weights each task uses and updates. Freezing a route learned on one task and evolving a new one for the next learned it faster than starting fresh or tweaking all the weights.

Abstract

For artificial general intelligence (AGI) it would be efficient if multiple users trained the same giant neural network, permitting parameter reuse, without catastrophic forgetting. PathNet is a first step in this direction. It is a neural network algorithm that uses agents embedded in the neural network whose task is to discover which parts of the network to re-use for new tasks. Agents are pathways (views) through the network which determine the subset of parameters that are used and updated by the forwards and backwards passes of the backpropogation algorithm. During learning, a tournament selection genetic algorithm is used to select pathways through the neural network for replication and mutation. Pathway fitness is the performance of that pathway measured according to a cost function. We demonstrate successful transfer learning; fixing the parameters along a path learned on task A and re-evolving a new population of paths for task B, allows task B to be learned faster than it could be learned from scratch or after fine-tuning. Paths evolved on task B re-use parts of the optimal path evolved on task A. Positive transfer was demonstrated for binary MNIST, CIFAR, and SVHN supervised learning classification tasks, and a set of Atari and Labyrinth reinforcement learning tasks, suggesting PathNets have general applicability for neural network training. Finally, PathNet also significantly improves the robustness to hyperparameter choices of a parallel asynchronous reinforcement learning algorithm (A3C).

Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, Daan Wierstra
arXiv:1701.08734 · cs.NE, cs.LG · submitted Jan 30, 2017
abstract · pdf · html

add comment on HN
Also discussed: Mar 2017 (1 point, 0 comments) · Feb 2017 (3 points, 0 comments)

In short, this architecture freezes the parameters and pathways used for previously learned tasks, and can learn new parameters and use new pathways for new tasks, with each new task learned faster than previous ones by leveraging all previously learned parameters and pathways (more efficient transfer learning).

It's a general neural net architecture.

Very cool.

"During learning, a tournament selection genetic algorithm is used to select pathways through the neural network for replication and mutation."

Trying to think of another 'tournament' like process that would allow for a massive distributed network where each node already has a decent GPU, where something like this could be successfully run. Maybe someone could help me out here...

I assume you're being sarcastic; they do point out in the intro and at the end that a deep RL agent could be trained to do the topology selections, but that would be more work to get going than some simple evolutionary operators, and is left to future work. Don't worry, I'm sure it'll be A3C all the way down eventually...
Well yes, you could use neural net for the tournament selection, but I was thinking of a much dumber competition that involves a whole lot more distributed GPU power.
I think you might have to actually explain what you're thinking of
loss function tournament winner replacing hash winner in something like bitcoin mining