about
Explanation for Existence of Adversarial Examples with Small Hamming Distance (arxiv.org)
3 points by DyslexicAtheist on Feb 6, 2019 | hide | past | pdf | discuss on HN

In plain words: Using the geometry of high-dimensional space, where distance counts how many input values you change, it shows why tiny edits can flip a network's decision. Targeted attacks should need to change about as many values as the network has classes, however deep it is.

Abstract · A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance

The existence of adversarial examples in which an imperceptible change in the input can fool well trained neural networks was experimentally discovered by Szegedy et al in 2013, who called them "Intriguing properties of neural networks". Since then, this topic had become one of the hottest research areas within machine learning, but the ease with which we can switch between any two decisions in targeted attacks is still far from being understood, and in particular it is not clear which parameters determine the number of input coordinates we have to change in order to mislead the network. In this paper we develop a simple mathematical framework which enables us to think about this baffling phenomenon from a fresh perspective, turning it into a natural consequence of the geometry of $\mathbb{R}^n$ with the $L_0$ (Hamming) metric, which can be quantitatively analyzed. In particular, we explain why we should expect to find targeted adversarial examples with Hamming distance of roughly $m$ in arbitrarily deep neural networks which are designed to distinguish between $m$ input classes.

Adi Shamir, Itay Safran, Eyal Ronen, Orr Dunkelman
arXiv:1901.10861 · cs.LG, cs.CR, stat.ML · submitted Jan 30, 2019
abstract · pdf · html

add comment on HN