about
Everybody dance now (arxiv.org)
2 points by alexghitza on Aug 27, 2018 | hide | past | pdf | discuss on HN

In plain words: It learns how the target person looks in each body pose from a few minutes of practice moves, then paints a dancer's poses onto them to make the target dance. The result looks convincing and smooth, and a tool can spot these fake clips.

Abstract · Everybody Dance Now

This paper presents a simple method for "do as I do" motion transfer: given a source video of a person dancing, we can transfer that performance to a novel (amateur) target after only a few minutes of the target subject performing standard moves. We approach this problem as video-to-video translation using pose as an intermediate representation. To transfer the motion, we extract poses from the source subject and apply the learned pose-to-appearance mapping to generate the target subject. We predict two consecutive frames for temporally coherent video results and introduce a separate pipeline for realistic face synthesis. Although our method is quite simple, it produces surprisingly compelling results (see video). This motivates us to also provide a forensics tool for reliable synthetic content detection, which is able to distinguish videos synthesized by our system from real data. In addition, we release a first-of-its-kind open-source dataset of videos that can be legally used for training and motion transfer.

Caroline Chan, Shiry Ginosar, Tinghui Zhou, Alexei A. Efros
arXiv:1808.07371 · cs.GR, cs.CV · submitted Aug 22, 2018 · updated Aug 27, 2019
abstract · pdf · html · In ICCV 2019

add comment on HN
Also discussed: Aug 2018 (2 points, 0 comments)