ML Newsnew | past | best | rss | submit about
Fooling surveillance cameras: adversarial patches to attack person detection (arxiv.org)
148 points by hardmaru on Apr 22, 2019 | hide | past | pdf | 23 comments on HN

In plain words: A printed pattern on a small cardboard plate, held in front of the body, tricks camera software into missing the person, even though people come in many shapes and poses. It greatly lowered detection accuracy and still worked when filmed by a real camera.

Abstract · Fooling automated surveillance cameras: adversarial patches to attack person detection

Adversarial attacks on machine learning models have seen increasing interest in the past years. By making only subtle changes to the input of a convolutional neural network, the output of the network can be swayed to output a completely different result. The first attacks did this by changing pixel values of an input image slightly to fool a classifier to output the wrong class. Other approaches have tried to learn "patches" that can be applied to an object to fool detectors and classifiers. Some of these approaches have also shown that these attacks are feasible in the real-world, i.e. by modifying an object and filming it with a video camera. However, all of these approaches target classes that contain almost no intra-class variety (e.g. stop signs). The known structure of the object is then used to generate an adversarial patch on top of it. In this paper, we present an approach to generate adversarial patches to targets with lots of intra-class variety, namely persons. The goal is to generate a patch that is able successfully hide a person from a person detector. An attack that could for instance be used maliciously to circumvent surveillance systems, intruders can sneak around undetected by holding a small cardboard plate in front of their body aimed towards the surveillance camera. From our results we can see that our system is able significantly lower the accuracy of a person detector. Our approach also functions well in real-life scenarios where the patch is filmed by a camera. To the best of our knowledge we are the first to attempt this kind of attack on targets with a high level of intra-class variety like persons.

Simen Thys, Wiebe Van Ranst, Toon Goedemé
arXiv:1904.08653 · cs.CV · submitted Apr 18, 2019
abstract · pdf · html · Accepted for CVPR Workshop: CV-COPS 2019

add comment on HN
Also discussed: Apr 2023 (3 points, 2 comments) · Apr 2019 (2 points, 0 comments)
ashleysmithgpu on Apr 22, 2019 [–]
Example video from the pdf: https://www.youtube.com/watch?v=MIbFvK2S9g8 Looks like it would work well on a tshirt
reply
ardy42 on Apr 22, 2019 [–]
Looking at the photos in the paper, that's the first ML-camouflage pattern that actually seems practical. I could see someone wearing a shirt with this "adversarial patch" looking totally unremarkable in public.

Other types of ML-camouflage, like https://cvdazzle.com/, make the wearer stand out far too much to actually be useful as anything but an art project.

reply
johnnycab on Apr 22, 2019 | parent [–]
There are other methods of occlusion and obfuscation to fool FR, varying in range of application and corresponding with the level of paranoia. The adversarial patch seems to fall somewhere along the lines of clothing that have images of faces or eyes on them to confuse FR until the algorithms eventually get better at recognising and countering them.

I don't what became of the 'infra-red baseball cap' discussed in the article. In my opinion, it not only seems like a different approach, but also feels innocuous, compared to some of the other methods.

https://www.wired.co.uk/article/avoid-facial-recognition-sof...

reply
jldugger on Apr 22, 2019 | parent [–]
I wonder though, seems like the fact that they chose a general purpose object classification implies that face detection still works.
reply
brk on Apr 22, 2019 | parent [–]
The "average" surveillance camera often does not have sufficient pixels-on-target/pixels-per-foot to do face detection for the majority of persons in the scene. The trend towards higher resolution cameras (eg: from an average resolution of ~3MP today towards 4K resolution) will of course help with face detection somewhat, but you also have angle of view concerns, and the fact that people don't always walk towards the camera for a good face shot.

Various companies have been implementing appearance search functionality that lets you search for similar appearances across multiple cameras to find the scene/image where you got a facial shot of a person of interest. Preventing the majority of cameras from classifying someone as a "person" object would significantly disrupt that workflow.

reply
mcbits on Apr 22, 2019 | parent [–]
Perhaps something like AdaBoost could find matches using a weak classifier on hundreds of low-resolution images (a few seconds of video) at different angles, assuming you can put together a similar data set for training... basically trading spatial information content for temporal.
reply
dsfyu404ed on Apr 22, 2019 [–]
The fact that the patch has to be placed around the waist seems to indicate that the shirt/pants transition is important to the algorithm (makes sense considering that training data full of normally dressed people is almost certainly going to have a lot of pictures of people that are dressed in a shirt that contrasts their pants). I wonder how if denim + denim, a coverall or a dress makes person detection harder.
reply
netwanderer3 on Apr 22, 2019 [–]
The demonstrating video was hilarious.

You cannot stop the camera to be there or aiming at you, so realistically the best option is to rely on a timeless principle: modifying input information will lead to changes of its output result, since the second element will always depend on the first.

I can see this is going to be another cat and mouse game in the near future. People are going to constantly come up with new creative ideas to circumvent around machine learning since it can only accurately identify data which it has already seen.

reply
greendestiny_re on Apr 22, 2019 [–]
Colorful eyewear can be used too.

https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf

Looks realistic, plausible and can be 3D printed (authors of the paper even provide the design sample).

reply
bsenftner on Apr 22, 2019 | parent [–]
This triggered interest in the FR research community, and then died as algorithms were tweaked to defeat the attack. Now, this is used as a filter to identify FR systems not keeping up.
reply
sandworm101 on Apr 22, 2019 [–]
This is not the first attempt at this. Anti-camera makeup has been a thing for a while. Painting a square on your nose or a flesh-colored eyepatch is often enough to fool facial rec systems looking to measure distances between features. Standing on a skateboard or moonwalking can sometimes trick a system into calling you a non-person. Putting a box over your head darlek-style works too.

None of these defeat old fashioned motion detection + followup by human eyes. Security guards arent going away anytime soon.

reply
starky on Apr 22, 2019 | parent [–]
Of course security guards aren't going anywhere anytime soon. Rather the new technology will make finding and acting on incidents quicker and more efficient.

I can't see it being too difficult to identify tactics being used and mark them as suspicious.

reply
JohnFen on Apr 22, 2019 | parent [–]
> I can't see it being too difficult to identify tactics being used and mark them as suspicious.

I'm sure you're right. But remember what every security measure is trying to achieve -- it's not actually making you immune to attack (that's impossible), but rather to increase the cost of the attack.

This sort of thing would accomplish that by requiring human involvement.

reply
sspencer on Apr 22, 2019 [–]
William Gibson had something like this in one of his later books. Spook Country maybe? As I recall it was simply called the Ugly Shirt.
reply
hindsightbias on Apr 22, 2019 [–]
Hinton is on it: https://heartbeat.fritz.ai/capsule-networks-a-new-and-attrac...
reply
yters on Apr 22, 2019 [–]
What about adversarial patches to make the camera ID a different targeted person? Could be used to frame someone.
reply
i_am_proteus on Apr 22, 2019 [–]
CV Dazzle, some earlier work in the same vein: https://ahprojects.com/cvdazzle/
reply
bsenftner on Apr 22, 2019 [–]
Eyeglass frames with eyes on them is a wonderful way to default FR.
reply
elijahtai on Apr 22, 2019 [–]
You can purchase a t-shirt from cloakwear.co
reply
OrgNet on Apr 22, 2019 | parent [–]
A Burqa[1] sounds like a safer bet (if you aren't carrying a cellphone).

1. https://duckduckgo.com/?q=burqa&t=ffab&iar=images&iax=images...

reply
deytempo on Apr 22, 2019 [–]
Half the IP cams out there are on a WiFi network that isn’t a dedicated vlan so an attacker could just ddos them or do a broadcast feedback loop to take them offline
reply
JohnFen on Apr 22, 2019 | parent [–]
A method that doesn't put you in legal jeopardy seems more desirable.
reply
deytempo on Apr 23, 2019 | parent [–]
I am merely stating a vulnerability not recommending an action
reply
About | RSS | RSS (all) | HN arXiv