about
Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis (arxiv.org)
2 points by PaulHoule on Jan 15, 2024 | hide | past | pdf | discuss on HN

In plain words: 3D Gaussian splats store a scene as thousands of soft colored blobs; this work shrinks them by grouping similar colors and shapes into shared low-bit tables. That cuts memory up to 31 times with barely any quality loss, and renders faster on small GPUs.

Abstract

Recently, high-fidelity scene reconstruction with an optimized 3D Gaussian splat representation has been introduced for novel view synthesis from sparse image sets. Making such representations suitable for applications like network streaming and rendering on low-power devices requires significantly reduced memory consumption as well as improved rendering efficiency. We propose a compressed 3D Gaussian splat representation that utilizes sensitivity-aware vector clustering with quantization-aware training to compress directional colors and Gaussian parameters. The learned codebooks have low bitrates and achieve a compression rate of up to $31\times$ on real-world scenes with only minimal degradation of visual quality. We demonstrate that the compressed splat representation can be efficiently rendered with hardware rasterization on lightweight GPUs at up to $4\times$ higher framerates than reported via an optimized GPU compute pipeline. Extensive experiments across multiple datasets demonstrate the robustness and rendering speed of the proposed approach.

Simon Niedermayr, Josef Stumpfegger, Rüdiger Westermann
arXiv:2401.02436 · cs.CV, cs.GR · submitted Nov 17, 2023 · updated Jan 22, 2024
abstract · pdf · html

add comment on HN