In plain words: Instead of retraining one network on each new task and losing old skills, it adds a stack of layers per task with sideways links to earlier stacks' features. On Atari and 3D maze games it beat the usual pretrain-then-finetune approach, reusing basic and high-level skills.
Abstract
Learning to solve complex sequences of tasks--while both leveraging transfer and avoiding catastrophic forgetting--remains a key obstacle to achieving human-level intelligence. The progressive networks approach represents a step forward in this direction: they are immune to forgetting and can leverage prior knowledge via lateral connections to previously learned features. We evaluate this architecture extensively on a wide variety of reinforcement learning tasks (Atari and 3D maze games), and show that it outperforms common baselines based on pretraining and finetuning. Using a novel sensitivity measure, we demonstrate that transfer occurs at both low-level sensory and high-level control layers of the learned policy.
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, Raia Hadsell
arXiv:1606.04671 · cs.LG · submitted Jun 15, 2016 · updated Oct 22, 2022
abstract · pdf · html