In plain words: Starting from only Gaussian distributions, it derives the equations diffusion models use to add and remove noise, along with likelihood formulas. The derivations recover the usual reverse-time equation and a family of noise-level-dependent variants without the variational step most textbooks rely on.
Abstract
This paper gives direct derivations of the differential equations and likelihood formulas of diffusion models assuming only knowledge of Gaussian distributions. A VAE analysis derives both forward and backward stochastic differential equations (SDEs) as well as non-variational integral expressions for likelihood formulas. A score-matching analysis derives the reverse diffusion ordinary differential equation (ODE) and a family of reverse-diffusion SDEs parameterized by noise level. The paper presents the mathematics directly with attributions saved for a final section.
David McAllester
arXiv:2301.11108 · cs.LG, cs.AI, math.PR · submitted Jan 25, 2023 · updated Mar 5, 2023
abstract · pdf · html