about
Softmax GAN (arxiv.org)
2 points by mayava on Apr 21, 2017 | hide | past | pdf | discuss on HN

In plain words: A fake-image generator swaps its usual real-or-fake scoring loss for a softmax, which spreads probability across all real and fake images in one batch. Real images soak up all the probability while fakes get none, and this swap makes training steadier than the original.

Abstract

Softmax GAN is a novel variant of Generative Adversarial Network (GAN). The key idea of Softmax GAN is to replace the classification loss in the original GAN with a softmax cross-entropy loss in the sample space of one single batch. In the adversarial learning of $N$ real training samples and $M$ generated samples, the target of discriminator training is to distribute all the probability mass to the real samples, each with probability $\frac{1}{M}$, and distribute zero probability to generated data. In the generator training phase, the target is to assign equal probability to all data points in the batch, each with probability $\frac{1}{M+N}$. While the original GAN is closely related to Noise Contrastive Estimation (NCE), we show that Softmax GAN is the Importance Sampling version of GAN. We futher demonstrate with experiments that this simple change stabilizes GAN training.

Min Lin
arXiv:1704.06191 · cs.LG, cs.NE · submitted Apr 20, 2017 · updated Jun 25, 2020
abstract · pdf · html · NIPS 2017 submission

add comment on HN