In plain words: A single fixed pattern pasted onto every image can hijack a trained classifier into doing a different task the attacker picks, like counting or reading digits. Unlike usual attacks that break the model or force one preset answer, this worked on six image recognizers.
Abstract
Deep neural networks are susceptible to \emph{adversarial} attacks. In computer vision, well-crafted perturbations to images can cause neural networks to make mistakes such as confusing a cat with a computer. Previous adversarial attacks have been designed to degrade performance of models or cause machine learning models to produce specific outputs chosen ahead of time by the attacker. We introduce attacks that instead {\em reprogram} the target model to perform a task chosen by the attacker---without the attacker needing to specify or compute the desired output for each test-time input. This attack finds a single adversarial perturbation, that can be added to all test-time inputs to a machine learning model in order to cause the model to perform a task chosen by the adversary---even if the model was not trained to do this task. These perturbations can thus be considered a program for the new task. We demonstrate adversarial reprogramming on six ImageNet classification models, repurposing these models to perform a counting task, as well as classification tasks: classification of MNIST and CIFAR-10 examples presented as inputs to the ImageNet model.
Gamaleldin F. Elsayed, Ian Goodfellow, Jascha Sohl-Dickstein
arXiv:1806.11146 · cs.LG, cs.CR, cs.CV, stat.ML · submitted Jun 28, 2018 · updated Nov 29, 2018
abstract · pdf · html
The idea here, at a very high level, is to use other neural nets that (a) accept different input data and learn to 'adversarially embed it' into the input data accepted by the first neural net, and (b) extract from the output of the first neural net the actual output desired by the attacker... without ever touching the first neural net.
The authors demonstrate adversarial neural nets that target a deep convnet trained to classify ImageNet data. They are able alter this convnet's function from ImageNet classification to counting squares in an image, classifying MNIST digits, and classifying CIFAR-10 images... without ever touching the convnet.
Great work.