In plain words: Instead of running a classifier over many candidate spots, one network looks at the whole image once and outputs boxes and labels directly. It runs in real time at 45 frames per second, with fewer false alarms than other detectors but more misplaced boxes.
Abstract
We present YOLO, a new approach to object detection. Prior work on object detection repurposes classifiers to perform detection. Instead, we frame object detection as a regression problem to spatially separated bounding boxes and associated class probabilities. A single neural network predicts bounding boxes and class probabilities directly from full images in one evaluation. Since the whole detection pipeline is a single network, it can be optimized end-to-end directly on detection performance. Our unified architecture is extremely fast. Our base YOLO model processes images in real-time at 45 frames per second. A smaller version of the network, Fast YOLO, processes an astounding 155 frames per second while still achieving double the mAP of other real-time detectors. Compared to state-of-the-art detection systems, YOLO makes more localization errors but is far less likely to predict false detections where nothing exists. Finally, YOLO learns very general representations of objects. It outperforms all other detection methods, including DPM and R-CNN, by a wide margin when generalizing from natural images to artwork on both the Picasso Dataset and the People-Art Dataset.
Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi
arXiv:1506.02640 · cs.CV · submitted Jun 8, 2015 · updated May 9, 2016
abstract · pdf · html
I also wonder to what extent merging the detection with underlying P-frame information from the video codecs would help. Knowing that a segment of video just moved to the left would mean the detected object could be moved to the left by the same amount, even if it was passing behind another object. Calculating the movement vectors independently seems silly if you can get that data from the underlying video codec itself.