In plain words: A neural network learns to turn per-pixel 3D scene details like position and surface direction into pixels with effects such as soft shadows and blur, instead of hand-written shading rules. It reaches similar quality and speed to expert-programmed shading while learning from example images.
Abstract · Deep Shading: Convolutional Neural Networks for Screen-Space Shading
In computer vision, convolutional neural networks (CNNs) have recently achieved new levels of performance for several inverse problems where RGB pixel appearance is mapped to attributes such as positions, normals or reflectance. In computer graphics, screen-space shading has recently increased the visual quality in interactive image synthesis, where per-pixel attributes such as positions, normals or reflectance of a virtual 3D scene are converted into RGB pixel appearance, enabling effects like ambient occlusion, indirect light, scattering, depth-of-field, motion blur, or anti-aliasing. In this paper we consider the diagonal problem: synthesizing appearance from given per-pixel attributes using a CNN. The resulting Deep Shading simulates various screen-space effects at competitive quality and speed while not being programmed by human experts but learned from example images.
Oliver Nalbach, Elena Arabadzhiyska, Dushyant Mehta, Hans-Peter Seidel, Tobias Ritschel
arXiv:1603.06078 · cs.GR, cs.LG · submitted Mar 19, 2016 · updated Aug 3, 2016
abstract · pdf · html