In plain words: Mixup trains a network on half-and-half blends of paired examples and their labels, so it learns to act smoothly between training points rather than memorizing them. It guessed better on new data than standard training, while copying wrong labels less and resisting trick inputs more.
Abstract · mixup: Beyond Empirical Risk Minimization
Large deep neural networks are powerful, but exhibit undesirable behaviors such as memorization and sensitivity to adversarial examples. In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their labels. By doing so, mixup regularizes the neural network to favor simple linear behavior in-between training examples. Our experiments on the ImageNet-2012, CIFAR-10, CIFAR-100, Google commands and UCI datasets show that mixup improves the generalization of state-of-the-art neural network architectures. We also find that mixup reduces the memorization of corrupt labels, increases the robustness to adversarial examples, and stabilizes the training of generative adversarial networks.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, David Lopez-Paz
arXiv:1710.09412 · cs.LG, stat.ML · submitted Oct 25, 2017 · updated Apr 27, 2018
abstract · pdf · html · ICLR camera ready version. Changes vs V1: fix repo URL; add ablation studies; add mixup + dropout etc