about
Meshtron: High-fidelity 3D mesh generation from point clouds (arxiv.org)
3 points by werediver on Dec 18, 2024 | hide | past | pdf | 1 comment on HN

In plain words: A tool that builds 3D surface shapes one face at a time, like an artist adding detail gradually, for games and animation. Earlier generators cap out around a thousand faces and coarse positions; this one makes shapes with up to 64,000 faces at finer detail.

Abstract · Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scale

Meshes are fundamental representations of 3D surfaces. However, creating high-quality meshes is a labor-intensive task that requires significant time and expertise in 3D modeling. While a delicate object often requires over $10^4$ faces to be accurately modeled, recent attempts at generating artist-like meshes are limited to $1.6$K faces and heavy discretization of vertex coordinates. Hence, scaling both the maximum face count and vertex coordinate resolution is crucial to producing high-quality meshes of realistic, complex 3D objects. We present Meshtron, a novel autoregressive mesh generation model able to generate meshes with up to 64K faces at 1024-level coordinate resolution --over an order of magnitude higher face count and $8{\times}$ higher coordinate resolution than current state-of-the-art methods. Meshtron's scalability is driven by four key components: (1) an hourglass neural architecture, (2) truncated sequence training, (3) sliding window inference, (4) a robust sampling strategy that enforces the order of mesh sequences. This results in over $50{\%}$ less training memory, $2.5{\times}$ faster throughput, and better consistency than existing works. Meshtron generates meshes of detailed, complex 3D objects at unprecedented levels of resolution and fidelity, closely resembling those created by professional artists, and opening the door to more realistic generation of detailed 3D assets for animation, gaming, and virtual environments.

Zekun Hao, David W. Romero, Tsung-Yi Lin, Ming-Yu Liu
arXiv:2412.09548 · cs.GR, cs.CV · submitted Dec 12, 2024
abstract · pdf · html · Project page: https://research.nvidia.com/labs/dir/meshtron/

add comment on HN

> ABSTRACT

> Meshes are fundamental representations of 3D surfaces. However, creating high- quality meshes is a labor-intensive task that requires significant time and expertise in 3D modeling. While a delicate object often requires over 104 faces to be accurately modeled, recent attempts at generating artist-like meshes are limited to 1.6K faces and heavy discretization of vertex coordinates. Hence, scaling both the maximum face count and vertex coordinate resolution is crucial to producing high-quality meshes of realistic, complex 3D objects. We present MESHTRON, a novel autoregressive mesh generation model able to generate meshes with up to 64K faces at 1024-level coordinate resolution –over an order of magnitude higher face count and 8× higher coordinate resolution than current state-of-the-art methods. MESHTRON’s scalability is driven by four key components: (i) an hourglass neural architecture, (ii) truncated sequence training, (iii) sliding window inference, and (iv) a robust sampling strategy that enforces the order of mesh sequences. This results in over 50% less training memory, 2.5× faster throughput, and better consistency than existing works. MESHTRON generates meshes of detailed, complex 3D objects at unprecedented levels of resolution and fidelity, closely resembling those created by professional artists, and opening the door to more realistic generation of detailed 3D assets for animation, gaming, and virtual environments.

Also featured in https://www.jendrikillner.com/post/graphics-programming-week...