ML Newsnew | past | best | rss | submit about
DensePose from WiFi (arxiv.org)
176 points by deverton on Jan 18, 2023 | hide | past | pdf | 44 comments on HN

In plain words: A system reads the phase and strength of WiFi signals and uses a neural network to map each body point into one of 24 regions. It tracked several people at once with accuracy close to camera methods, needing no camera or costly sensor.

Abstract · DensePose From WiFi

Advances in computer vision and machine learning techniques have led to significant development in 2D and 3D human pose estimation from RGB cameras, LiDAR, and radars. However, human pose estimation from images is adversely affected by occlusion and lighting, which are common in many scenarios of interest. Radar and LiDAR technologies, on the other hand, need specialized hardware that is expensive and power-intensive. Furthermore, placing these sensors in non-public areas raises significant privacy concerns. To address these limitations, recent research has explored the use of WiFi antennas (1D sensors) for body segmentation and key-point body detection. This paper further expands on the use of the WiFi signal in combination with deep learning architectures, commonly used in computer vision, to estimate dense human pose correspondence. We developed a deep neural network that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions. The results of the study reveal that our model can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches, by utilizing WiFi signals as the only input. This paves the way for low-cost, broadly accessible, and privacy-preserving algorithms for human sensing.

Jiaqi Geng, Dong Huang, Fernando De la Torre
arXiv:2301.00250 · cs.CV · submitted Dec 31, 2022
abstract · pdf · html · 13 pages, 10 figures

add comment on HN
Also discussed: Jan 2023 (1 point, 0 comments) · Jan 2023 (4 points, 1 comment)
transpute on Jan 18, 2023 [–]
IEEE 802.11bf aims to standardize through-wall motion sensing for Wi-Fi 7 in 2024, https://www.originwirelessai.com/insights/what-is-the-802-11... & https://www.originwirelessai.com/insights/origins-next-chapt.... It can be done today with custom firmware on low-cost WiFi devices. Over the past decade, hundreds of Wi-Fi sensing papers have been published, for different categories of motion.

> capture tiny, vital movements of human breathing, even when people are not in the line-of-sight of a device ... whole home coverage to detect falls, breathing rates, abnormal behaviors ... Internet Service Providers (ISPs), Security, Health & Wellness, Automotive, and IoT

The images in the DensePose paper are helpful: https://twitter.com/aibreakfast/status/1613550599144091650

Prior HN threads on Wi-Fi 7 Sensing: https://news.ycombinator.com/item?id=33758434

Counter-measures: https://news.ycombinator.com/item?id=27121918#27133079

reply
pontifier on Jan 18, 2023 | parent [–]
Sign me up for the security system. This seems like a total game changer if it's possible to do real human detection and tracking in a secure location.
reply
transpute on Jan 18, 2023 | parent [–]
2012, https://www.popsci.com/technology/article/2012-07/seeing-thr...

> The U.K. Ministry of Defence has funded a study to determine whether it could be used to scan buildings during urban warfare. With improvements, Woodbridge says, the device could become sensitive enough to pick up on subtle motions the ribcage makes during breathing, which would allow the radar to detect people who are standing or sitting still.

reply
actionfromafar on Jan 18, 2023 | parent [–]
Enter some weird cyber-defense suit which masks bodily movements when you sit still...
reply
yencabulator on Jan 19, 2023 | parent [–]
Sounds more like a job for just plain old RF jamming.
reply
XorNot on Jan 18, 2023 | parent [–]
I have a weirder utility for it: occupancy sensing in rooms.

If it can detect breathing of humans reliably, then it solves a huge problem in having home automation with automatic lighting - particularly in bathrooms. I've never had a decent bathroom occupancy sensor (they all end up wanting to detect fairly large motions) - the obvious solution is AI with a camera but for obvious reasons no, but if a couple of base stations can localize person positions to rooms in the house (and provide other services) then that kind of solves the whole issue!

reply
nine_k on Jan 18, 2023 | parent [–]
Is there anything wrong with mechanical switches, beside the apparent (though negligible for me) improvement in comfort?
reply
XorNot on Jan 18, 2023 | parent [–]
I grew up with a father who was obsessed with the idea that we were wasting electricity by accidentally leaving the lights on sometimes. As a result he's spent probably thousands in attempted energy saving devices which have all not worked and definitely would never save enough compared to the cost of them.

Which is why I think about these things despite this seemingly obvious solution.

And the fact that the guest bathroom when I visit his house still has no light switch but does have a cheap motion sensor which is annoying as hell.

reply
eightysixfour on Jan 18, 2023 | parent [–]
There are a number of mmWave occupancy sensors that are more than capable of this. The biggest issue is the delay introduced in most smart home contexts by Zigbee and the hubs. Matter/Thread should help here.

Existing options: https://smarthomescene.com/reviews/best-mmwave-human-presenc...

The Aqara FP2 should be out in Q2 with Matter support and be a nice upgrade. It can segment a room into multiple spaces and detect 5 people.

reply
hooverd on Jan 18, 2023 | parent [–]
"Please drink verification can to continue."
reply
benob on Jan 18, 2023 [–]
From the abstract: "This paves the way for low-cost, broadly accessible, and privacy-preserving algorithms for human sensing."

How much do we have to wait for the follow up paper: "Human identification through wifi signals" and then the patent for "Wifi as an identity tracking source for advertisement in public spaces"

Technology advances make me sad those days...

reply
transpute on Jan 18, 2023 | parent [–]
> How much do we have to wait for the follow up paper

Early work from ~2010, https://dhalperi.github.io/linux-80211n-csitool/ & https://groups.csail.mit.edu/netmit/sFFT/soda_paper.pdf & https://dspace.mit.edu/handle/1721.1/103715

1000+ papers, add your surveillance keyword and 'Search within citing articles': https://scholar.google.com/scholar?cites=4354757827101869878

https://en.wikipedia.org/wiki/Dina_Katabi

> In 2012, her work on Sparse Fourier Transforms was chosen as one of the top 10 breakthroughs of the year by Technology Review ... In 2014, on the celebration of Project Mac's 50th anniversary, her work on X-ray vision was chosen as one of the "50 ways that MIT has transformed computer science."

Sample paper topic keywords, http://web.archive.org/web/20210620035520/http://dhalperi.gi...

  human-to-human interaction recognition
  device-free human activity recognition
  occupant activity recognition in smart offices
  emotion sensing via wireless channel data
  CSI learning for gait biometric sensing
  sleep monitoring from afar 
  human breath status via commodity wifi
  device-free crowd sensing
reply
sitkack on Jan 18, 2023 | parent [–]
Measuring sleep position with wireless signals https://people.csail.mit.edu/scyue/projects/bodycompass/
reply
fortran77 on Jan 18, 2023 | parent [–]
You can distinguish people by their gait, so it may be plausible that measurements of gross movements can pick a person out.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3925574/

reply
moritonal on Jan 18, 2023 | parent [–]
Wouldn't gait tracking be the obvious next step?
reply
jerpint on Jan 18, 2023 | parent [–]
I was thinking the same thing
reply
danuker on Jan 18, 2023 | parent [–]
> Human identification through wifi signals

It is already easy to track anyone who owns a smartphone.

Probe requests broadcast the MAC address: https://medium.com/@brannondorsey/wi-fi-is-broken-3f6054210f...

reply
transpute on Jan 18, 2023 | parent [–]
Wi-Fi Sensing can potentially enable identification of humans carrying _no_ devices.
reply
app4soft on Jan 23, 2023 | parent [–]
> Probe requests broadcast the MAC address: https://medium.com/@brannondorsey/wi-fi-is-broken-3f6054210f...

Summary is good:

> Wi-Fi is fucked, everything is broken, Donald J. Trump is president, and the earth is dying. Ok, taking a step back, you might be wondering what you can do to protect yourself. The solution is not without flaws; turn your Wi-Fi off when you aren’t connected to a known network. Doing so will prevent your device from leaking your network names and device fingerprint to the open world. The solution is awkward, easy to forget, and sub-par. But its what we’ve got.

reply
fortran77 on Jan 18, 2023 | parent [–]
That's why many vendors randomize MAC addresses, at least where and when possible.
reply
skibidibipiti on Jan 20, 2023 | parent [–]
Cmon, use your imagination, It’s worse than that. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6513191/
reply
taylorius on Jan 18, 2023 | parent [–]
I'm not sure "Privacy preserving" is the phrase that springs to mind with this tech.
reply
carbocation on Jan 18, 2023 [–]
> This paves the way for low-cost, broadly accessible, and privacy-preserving algorithms for human sensing.

Just because this surveillance tech avoids photons with energies accessible to human photoreceptors doesn’t mean it preserves privacy.

reply
sitkack on Jan 18, 2023 [–]
Take a look at Figure 10, mind blowing.

This tech will make it into every device with an RF receiver. Your TV, phone, router, computer, tablet, fridge, lightbulb will know your pulse and waste size at all locations in your house.

reply
nathancahill on Jan 18, 2023 | parent [–]
I think you meant waist size, but the typo is even more privacy invasive.
reply
sitkack on Jan 19, 2023 | parent [–]
My bad, I meant waste volume.
reply
transpute on Jan 18, 2023 | parent [–]
Plus keyboard-entered passwords, thanks to mmWave radar being added to the Wi-Fi spec, https://news.ycombinator.com/item?id=30172647
reply
GordonS on Jan 18, 2023 [–]
This could be a boon for thieves, allowing them to scan for empty houses at night, so they know which ones can be more safely targeted.
reply
jerpint on Jan 18, 2023 | parent [–]
It can also serve as a replacement for IR sensors in homes and keep thieves at bay
reply
kachnuv_ocasek on Jan 18, 2023 | parent [–]
Who do you think will be faster in adopting the technology—burglars or homeowners?
reply
throw10920 on Jan 18, 2023 | parent [–]
Moreover, thieves are likely to use custom-built RF occupancy detection devices, while the vast majority of homeowners will buy an off-the-shelf product that will either sell their data to advertisers (a la "smart" televisions), require a subscription service, or both.
reply
bemmu on Jan 18, 2023 [–]
When we have near-human level deep learning models for things humans can do, what does this level of performance look like when applied to things we can’t do at all? This is a nice example of this.
reply
thom on Jan 18, 2023 [–]
The side-channel attacks are coming from inside the house.
reply
fulafel on Jan 18, 2023 [–]
What kind of measures for privacy can be implemented for this? Technically or regulation wise.
reply
transpute on Jan 18, 2023 | parent [–]
Technical defenses are either cat & mouse CSI electronic warfare or shielding the walls+floor+roof+windows of some rooms (e.g. research labs in business, industrial systems, WFH offices, bedrooms) with materials like aluminum radiant barrier, RF window film or expensive drywall (e.g. QuietRock) designed for RF shielding of SCIFs.

Another defense, impractical for most people, is for their house/office to be physically protected by a tall fence that is far enough away from the building to exceed radio range, along with sensors at the fence to detect someone attempting to broadcast into the property. This could be combined with shielding to reduce the necessary fence-building gap.

Regulatory responses: some countries could restrict this technology to high-frequency 60Ghz mmWave which doesn't easily pass through walls. That would at least reduce the social issue of X-ray vision (via 2.4Ghz WiFi) through the exterior walls of existing homes and businesses.

One of the 802.11bf papers proposed an opt-out mechanism where human biometrics could be registered into a giant database and you could ask WiFi devices to unsee some humans. Good luck with that.

Another regulatory response could be to delay approval of Wi-Fi 7 and shipment of Wi-Fi Sensing devices until the WiFi industry pays to upgrade all existing houses with RF shielding, i.e. never. Maybe it would be sufficient to wait until the homes of celebrities and politicians were RF shielded? Let's not forget IP-sensitive and financial business offices with people typing on keyboards. And using combination locks. And...

reply
nine_k on Jan 18, 2023 | parent [–]
I wonder how much shielding is actually needed. 2.4 GHz = 12.5 cm wave. A conductive grid with approximately 6cm (or 2") step should prevent the waves from getting inside. Not a very expensive thing to add even to drywall panels. Can also be relatively easily added into fabrics, to make RF-reflective curtains.
reply
orbital-decay on Jan 18, 2023 | parent [–]
Some active device equivalent to a mouse jiggler, retransmitting the signal and shifting the phase just enough to create the illusion of constant movement.
reply
jpttsn on Jan 18, 2023 | parent [–]
Regulation wise we can hope the EU forces us to click through accept popups whenever we walk into a room.
reply
bick_nyers on Jan 18, 2023 [–]
Absolutely incredible, the applications of machine learning never cease to amaze me.

Will be an absolute boon for VR.

reply
jimmySixDOF on Jan 18, 2023 | parent [–]
Interesting. Full Body Tracking is one of the few places I can see this getting real end user traction considering how hard it is to do now but a lot of work goes into the pose estimation side and will need to be integrated to the AP. A Modified Microsoft Kinect or Intel Realsense built into their own SoC then it's APIs out to the VR Headset. And at that point it's not just an Access Point anymore.
reply
jayd16 on Jan 18, 2023 | parent [–]
>Will be an absolute boon for VR.

Ehhh... The latency would need to get lower than camera based systems but it might be useful to some degree.

reply
gene-h on Jan 18, 2023 [–]
It doesn't work well if the layout of wifi transceivers it was trained on changes
reply
zachthewf on Jan 18, 2023 [–]
Interesting. Could be valuable for fall detection without requiring a smart watch.
reply
lowbloodsugar on Jan 18, 2023 [–]
Why does this sound like exposition from a dystopian Neal Asher trilogy?
reply
About | RSS | RSS (all) | HN arXiv