In plain words: Two neural networks play an opposing game: one makes outputs, the other predicts their effects, each trying to win. This shows image generators are a special case of that 1990 idea, and corrects an old claim that a 1990s rival wasn't such a game.
Abstract · Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)
I review unsupervised or self-supervised neural networks playing minimax games in game-theoretic settings: (i) Artificial Curiosity (AC, 1990) is based on two such networks. One network learns to generate a probability distribution over outputs, the other learns to predict effects of the outputs. Each network minimizes the objective function maximized by the other. (ii) Generative Adversarial Networks (GANs, 2010-2014) are an application of AC where the effect of an output is 1 if the output is in a given set, and 0 otherwise. (iii) Predictability Minimization (PM, 1990s) models data distributions through a neural encoder that maximizes the objective function minimized by a neural predictor of the code components. I correct a previously published claim that PM is not based on a minimax game.
Juergen Schmidhuber
arXiv:1906.04493 · cs.NE, cs.LG · submitted Jun 11, 2019 · updated Apr 22, 2020
abstract · pdf · html · 15 pages, 1 figure, 104 references
and https://twitter.com/hardmaru/status/1138678311884738560