about
MixMatch: A Holistic Approach to Semi-Supervised Learning (arxiv.org)
4 points by pplonski86 on May 9, 2019 | hide | past | pdf | discuss on HN

In plain words: MixMatch trains on unlabeled images by guessing confident labels for slightly altered copies, then blends labeled and unlabeled examples together during training. With just 250 labels on CIFAR-10 it cut errors from 38% to 11%, far better than the usual semi-supervised approach.

Abstract

Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. In this work, we unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that works by guessing low-entropy labels for data-augmented unlabeled examples and mixing labeled and unlabeled data using MixUp. We show that MixMatch obtains state-of-the-art results by a large margin across many datasets and labeled data amounts. For example, on CIFAR-10 with 250 labels, we reduce error rate by a factor of 4 (from 38% to 11%) and by a factor of 2 on STL-10. We also demonstrate how MixMatch can help achieve a dramatically better accuracy-privacy trade-off for differential privacy. Finally, we perform an ablation study to tease apart which components of MixMatch are most important for its success.

David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, Colin Raffel
arXiv:1905.02249 · cs.LG, cs.AI, cs.CV, stat.ML · submitted May 6, 2019 · updated Oct 23, 2019
abstract · pdf · html

add comment on HN