about
Re-Rend: Real-Time Rendering of NeRFs Across Devices (arxiv.org)
3 points by prtg on Oct 9, 2023 | hide | past | pdf | discuss on HN

In plain words: A trained NeRF is converted into a mesh for shape plus simple matrices for color, so an ordinary graphics chip can draw each pixel with one cheap lookup instead of hundreds of network queries. It renders over 2.6 times faster than the best prior real-time approach with no visible drop in image quality.

Abstract · Re-ReND: Real-time Rendering of NeRFs across Devices

This paper proposes a novel approach for rendering a pre-trained Neural Radiance Field (NeRF) in real-time on resource-constrained devices. We introduce Re-ReND, a method enabling Real-time Rendering of NeRFs across Devices. Re-ReND is designed to achieve real-time performance by converting the NeRF into a representation that can be efficiently processed by standard graphics pipelines. The proposed method distills the NeRF by extracting the learned density into a mesh, while the learned color information is factorized into a set of matrices that represent the scene's light field. Factorization implies the field is queried via inexpensive MLP-free matrix multiplications, while using a light field allows rendering a pixel by querying the field a single time-as opposed to hundreds of queries when employing a radiance field. Since the proposed representation can be implemented using a fragment shader, it can be directly integrated with standard rasterization frameworks. Our flexible implementation can render a NeRF in real-time with low memory requirements and on a wide range of resource-constrained devices, including mobiles and AR/VR headsets. Notably, we find that Re-ReND can achieve over a 2.6-fold increase in rendering speed versus the state-of-the-art without perceptible losses in quality.

Sara Rojas, Jesus Zarzar, Juan Camilo Perez, Artsiom Sanakoyeu, Ali Thabet, Albert Pumarola, Bernard Ghanem
arXiv:2303.08717 · cs.CV, cs.GR · submitted Mar 15, 2023
abstract · pdf · html

add comment on HN