ML Newsnew | past | best | rss | submit about
Fast and accurate object detection in high resolution 4K and 8K video using GPUs (arxiv.org)
61 points by joubert on Dec 27, 2018 | hide | past | pdf | 17 comments on HN

In plain words: It first checks a rough version of each frame to spot objects, then re-examines only those spots at full sharpness. Accuracy stays high while speed reaches 3-6 frames per second on 4K and 2 on 8K, without checking every pixel at full detail.

Abstract

Machine learning has celebrated a lot of achievements on computer vision tasks such as object detection, but the traditionally used models work with relatively low resolution images. The resolution of recording devices is gradually increasing and there is a rising need for new methods of processing high resolution data. We propose an attention pipeline method which uses two staged evaluation of each image or video frame under rough and refined resolution to limit the total number of necessary evaluations. For both stages, we make use of the fast object detection model YOLO v2. We have implemented our model in code, which distributes the work across GPUs. We maintain high accuracy while reaching the average performance of 3-6 fps on 4K video and 2 fps on 8K video.

Vít Růžička, Franz Franchetti
arXiv:1810.10551 · cs.CV, cs.LG · submitted Oct 24, 2018
abstract · pdf · html · 6 pages, 12 figures, Best Paper Finalist at IEEE High Performance Extreme Computing Conference (HPEC) 2018; copyright 2018 IEEE; (DOI will be filled when known)

add comment on HN

reaperducer on Dec 27, 2018 [–]
Somewhat OT, but standing in line at the grocery store checkout last night I couldn't help but wonder why so many billions are being spent on complex visual problems like self-driving cars, then at the same time the checkout clerk has to look in a laminated binder and then manually punch in a code number to get a price for bananas. (Happens at both big chains around here — Safeway and Kroger alliances.)

Surely with all this image AI and ML coming out of SV, there must be a way to augment cash registers to detect the difference between a dragonfruit and a kumquat.

reply
magnetic on Dec 27, 2018 | parent [–]
Looking at the laminated binder is not a nominal case, thankfully. Think of it as a cache miss.

Most checkout clerks have memorized the codes and don't refer to the binder anymore. Perhaps you saw a new employee.

Also, many fruits/veggies have stickers on them with the code (either numeric or bar-code) so you don't have to do a lookup into "the binder". That's how you can easily go through the "self checkout" lanes (although there is a fallback in the form of a "virtual binder" in the UI if you can't find the code and want to select your produce from a set of buttons/images on the screen, with optional string searches).

It's often key to have a differentiator in the form of a code because different items can be visually equivalent (think about bananas vs organic bananas).

reply
reaperducer on Dec 27, 2018 | parent [–]
many fruits/veggies have stickers on them with the code

IME, too often the sticker isn't useful because it's fallen off, or is obscured by the green eco-friendly produce bags.

That's how you can easily go through the "self checkout" lanes

I don't use self-checkout, on principle. I'd rather wait in line three minutes than cost a minimum-wage checkout person their job.

It's often key to have a differentiator in the form of a code because different items can be visually equivalent (think about bananas vs organic bananas).

This is an excellent point I didn't think of. I don't know that AI will be able to distinguish regular from organic produce in my lifetime.

reply
gattilorenz on Dec 28, 2018 | parent [–]
OT, but...

> I don't use self-checkout, on principle. I'd rather wait in line three minutes than cost a minimum-wage checkout person their job

Thanks for stating this, I never realized the potential connection between the two. I'm going to do the same thing from now on.

reply
gjm11 on Dec 27, 2018 | parent [–]
> I don't know that AI will be able to distinguish regular from organic produce in my lifetime.

Yeah, it will. The not-organic produce will have been genetically engineered to contain RFID tags :-).

reply
joubert on Dec 27, 2018 | parent [–]
Amazon Go store - https://www.analyticsvidhya.com/blog/2018/01/amazon-go-ai-po...

Perhaps we’ll see more of the tech you mention in Whole Foods and other brick and mortar businesses that Amazon is getting into.

reply
hasseio on Dec 28, 2018 | parent [–]
In Australia the 'binder' is builtin to the cash register. A simple touch interface allows the checkout clerk to select the vege/fruit that currently being weighted on the scanning platform.
reply
Razengan on Dec 28, 2018 | parent [–]
> I couldn't help but wonder why so many billions are being spent

I wonder the same about why shops need to be manned at all.

Vending machines have been a solved problem since many decades.

Just supersize them. A secure warehouse connected to a few external despensers won’t even take up as much space as a full brick-and-mortar store that people need to walk around in.

Chalk this up as another example of things that should have been commonplace 20 years ago but didn’t.

reply
lj3 on Dec 27, 2018 | parent [–]
Human labor is still cheaper and more consistent than current AI and ML systems.
reply
devonkim on Dec 27, 2018 | parent [–]
The issue of human labor cost isn’t the whole picture as much as the TCO of an impoverished employee class such as high employee theft rates, lost work days from stress / poor nutrition / commute / lack of energy for exercise. Then there’s also the PR and legal necessary to avoid paying out fraudulent worker’s comp cases. There’s a lot of overhead that comes from treating employees like cattle that it makes me wonder if things may be better by just paying them more.
reply
gnarbarian on Dec 27, 2018 | parent [–]
Ahh but could ML spot the difference between an organic banana and a regular one? I bet it could but it would be trickier.
reply
mobilemidget on Dec 27, 2018 | parent [–]
For sure it will recognise fruits and vegetables better then any teenager around here that has a job as cashier. Amount of times I had to facepalm because cashier didn't recognise a beet or other common vegetable or fruit is significant.

But I refuse to self scan until you have to pay to use a cashier or self scan gets you a discount.

reply
reaperducer on Dec 27, 2018 | parent [–]
Amount of times I had to facepalm because cashier didn't recognise a beet or other common vegetable or fruit is significant.

Yeah, I have that problem, too. And I don't even eat "exotic"-looking things like pomegranates. Finger potatoes, kiwis, beets, cherry tomatoes (are they cherries or tomatoes?), and lots of other things confound grocery clerks these days.

And holy cow, don't try to pay with a two-dollar bill, or a dollar coin!

But I refuse to self scan until you have to pay to use a cashier or self scan gets you a discount.

Agreed.

reply
bcatanzaro on Dec 27, 2018 [–]
They used P100 GPUs that have a peak throughout of around 10 TFlops. V100 GPUs, or the RTX 2080 Ti/RTX Titan, have north of 100 TFlops.

So it’d be interesting for them to update their performance figures with current GPUs. It likely would run at much higher frame rates.

reply
ingenieroariel on Dec 27, 2018 [–]
I have six gopro in an spherical arrangement and each one produces 960p video at 120fps, this renders a spherical 4k video.

From what I read they can process Yolo 6fps on multiple GPUs. Is a 20x speedup with more gpu possible based on their architecture?

reply
CorvusCrypto on Dec 27, 2018 | parent [–]
Well for training attention-cnn model I think it.would speed it up, but as they note in the paper, running the pipeline they experienced worker saturation on their dataset (see figure 12). Looks like saturation occurs after 7 parallel workers with their hardware.
reply
nshm on Dec 28, 2018 [–]
Just 4 frames per second sounds crazy slow.
reply
About | RSS | RSS (all) | HN arXiv