about
On the Mathematics of Diffusion Models (arxiv.org)
17 points by panabee on Feb 21, 2023 | hide | past | pdf | discuss on HN

In plain words: Starting from only Gaussian distributions, it derives the equations diffusion models use to add and remove noise, along with likelihood formulas. The derivations recover the usual reverse-time equation and a family of noise-level-dependent variants without the variational step most textbooks rely on.

Abstract

This paper gives direct derivations of the differential equations and likelihood formulas of diffusion models assuming only knowledge of Gaussian distributions. A VAE analysis derives both forward and backward stochastic differential equations (SDEs) as well as non-variational integral expressions for likelihood formulas. A score-matching analysis derives the reverse diffusion ordinary differential equation (ODE) and a family of reverse-diffusion SDEs parameterized by noise level. The paper presents the mathematics directly with attributions saved for a final section.

David McAllester
arXiv:2301.11108 · cs.LG, cs.AI, math.PR · submitted Jan 25, 2023 · updated Mar 5, 2023
abstract · pdf · html

add comment on HN