about
ReVision: Video Generation with Explicit 3D Physics Modeling for Complex Motion (arxiv.org)
1 point by badmonster on May 2, 2025 | hide | past | pdf | discuss on HN

In plain words: ReVision makes a rough video first, then reads 3D shapes and motion from it, corrects the motion with a learned 3D model, and feeds it to the same generator to redraw the video. Its 1.5B-parameter setup beat a 13B model on complex actions and interactions.

Abstract · ReVision: Refining Video Diffusion with Explicit 3D Motion Modeling

In recent years, video generation has seen significant advancements. However, challenges still persist in generating complex motions and interactions. To address these challenges, we introduce ReVision, a plug-and-play framework that explicitly integrates parameterized 3D model knowledge into a pretrained conditional video generation model, significantly enhancing its ability to generate high-quality videos with complex motion and interactions. Specifically, ReVision consists of three stages. First, a video diffusion model is used to generate a coarse video. Next, we extract a set of 2D and 3D features from the coarse video to construct a 3D object-centric representation, which is then refined by our proposed parameterized motion prior model to produce an accurate 3D motion sequence. Finally, this refined motion sequence is fed back into the same video diffusion model as additional conditioning, enabling the generation of motion-consistent videos, even in scenarios involving complex actions and interactions. We validate the effectiveness of our approach on Stable Video Diffusion, where ReVision significantly improves motion fidelity and coherence. Remarkably, with only 1.5B parameters, it even outperforms a state-of-the-art video generation model with over 13B parameters on complex video generation by a substantial margin. Our results suggest that, by incorporating 3D motion knowledge, even a relatively small video diffusion model can generate complex motions and interactions with greater realism and controllability, offering a promising solution for physically plausible video generation.

Qihao Liu, Ju He, Qihang Yu, Liang-Chieh Chen, Alan Yuille
arXiv:2504.21855 · cs.CV · submitted Apr 30, 2025 · updated Jan 8, 2026
abstract · pdf · html · TMLR camera-ready version. Project Page: https://revision-video.github.io/

add comment on HN