about
NeMF: Neural Motion Fields for Kinematic Animation (arxiv.org)
4 points by lnyan on Jun 8, 2022 | hide | past | pdf | discuss on HN

In plain words: Instead of storing a motion as a list of separate frames, this system learns one continuous function that outputs a pose at any moment in time, with a random vector picking the style. It beat frame-by-frame methods at filling in, blending, and redirecting motions.

Abstract

We present an implicit neural representation to learn the spatio-temporal space of kinematic motions. Unlike previous work that represents motion as discrete sequential samples, we propose to express the vast motion space as a continuous function over time, hence the name Neural Motion Fields (NeMF). Specifically, we use a neural network to learn this function for miscellaneous sets of motions, which is designed to be a generative model conditioned on a temporal coordinate $t$ and a random vector $z$ for controlling the style. The model is then trained as a Variational Autoencoder (VAE) with motion encoders to sample the latent space. We train our model with a diverse human motion dataset and quadruped dataset to prove its versatility, and finally deploy it as a generic motion prior to solve task-agnostic problems and show its superiority in different motion generation and editing applications, such as motion interpolation, in-betweening, and re-navigating. More details can be found on our project page: https://cs.yale.edu/homes/che/projects/nemf/.

Chengan He, Jun Saito, James Zachary, Holly Rushmeier, Yi Zhou
arXiv:2206.03287 · cs.CV, cs.GR · submitted Jun 4, 2022 · updated Oct 9, 2022
abstract · pdf · html · Accepted to NeurIPS 2022. Project page: https://cs.yale.edu/homes/che/projects/nemf/

add comment on HN