about
Volumetric Rendering with Baked Quadrature Fields (arxiv.org)
1 point by PaulHoule on Dec 12, 2023 | hide | past | pdf | discuss on HN

In plain words: Instead of sampling a scene along a ray, this builds a polygon mesh whose surfaces mark the sample points, with textures holding color and opacity so a graphics card can draw it. It renders 1080p at over 100 frames per second while showing see-through effects.

Abstract

We propose a novel Neural Radiance Field (NeRF) representation for non-opaque scenes that enables fast inference by utilizing textured polygons. Despite the high-quality novel view rendering that NeRF provides, a critical limitation is that it relies on volume rendering that can be computationally expensive and does not utilize the advancements in modern graphics hardware. Many existing methods fall short when it comes to modelling volumetric effects as they rely purely on surface rendering. We thus propose to model the scene with polygons, which can then be used to obtain the quadrature points required to model volumetric effects, and also their opacity and colour from the texture. To obtain such polygonal mesh, we train a specialized field whose zero-crossings would correspond to the quadrature points when volume rendering, and perform marching cubes on this field. We then perform ray-tracing and utilize the ray-tracing shader to obtain the final colour image. Our method allows an easy integration with existing graphics frameworks allowing rendering speed of over 100 frames-per-second for a $1920\times1080$ image, while still being able to represent non-opaque objects.

Gopal Sharma, Daniel Rebain, Kwang Moo Yi, Andrea Tagliasacchi
arXiv:2312.02202 · cs.GR, cs.CV · submitted Dec 2, 2023 · updated Jul 10, 2024
abstract · pdf · html

add comment on HN