about
Gen4D: Synthesizing Humans and Scenes in the Wild (arxiv.org)
1 point by PaulHoule on Jun 26, 2025 | hide | past | pdf | discuss on HN

In plain words: A pipeline turns recorded expert motions and text prompts into lifelike moving people placed in matching backgrounds, all automatically. Unlike hand-built asset libraries and manual 3D scene design, it needs no manual modeling and produced a dataset covering three sports.

Abstract

Lack of input data for in-the-wild activities often results in low performance across various computer vision tasks. This challenge is particularly pronounced in uncommon human-centric domains like sports, where real-world data collection is complex and impractical. While synthetic datasets offer a promising alternative, existing approaches typically suffer from limited diversity in human appearance, motion, and scene composition due to their reliance on rigid asset libraries and hand-crafted rendering pipelines. To address this, we introduce Gen4D, a fully automated pipeline for generating diverse and photorealistic 4D human animations. Gen4D integrates expert-driven motion encoding, prompt-guided avatar generation using diffusion-based Gaussian splatting, and human-aware background synthesis to produce highly varied and lifelike human sequences. Based on Gen4D, we present SportPAL, a large-scale synthetic dataset spanning three sports: baseball, icehockey, and soccer. Together, Gen4D and SportPAL provide a scalable foundation for constructing synthetic datasets tailored to in-the-wild human-centric vision tasks, with no need for manual 3D modeling or scene design.

Jerrin Bright, Zhibo Wang, Yuhao Chen, Sirisha Rambhatla, John Zelek, David Clausi
arXiv:2506.05397 · cs.GR, cs.AI · submitted Jun 3, 2025
abstract · pdf · html · Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

add comment on HN