In plain words: A Julia renderer draws 3D scenes as images and automatically works out how to tweak the scene so the picture matches a target, instead of only drawing forward. It rebuilt scenes from images and paired with neural networks.
Abstract · RayTracer.jl: A Differentiable Renderer that supports Parameter Optimization for Scene Reconstruction
In this paper, we present RayTracer.jl, a renderer in Julia that is fully differentiable using source-to-source Automatic Differentiation (AD). This means that RayTracer not only renders 2D images from 3D scene parameters, but it can be used to optimize for model parameters that generate a target image in a Differentiable Programming (DP) pipeline. We interface our renderer with the deep learning library Flux for use in combination with neural networks. We demonstrate the use of this differentiable renderer in rendering tasks and in solving inverse graphics problems.
Avik Pal
arXiv:1907.07198 · cs.GR, cs.CV · submitted Jul 16, 2019 · updated Nov 1, 2019
abstract · pdf · html · Proceedings of the JuliaCon Conferences 2019