about
Generalized Discrete Diffusion from Snapshots (arxiv.org)
1 point by GaggiX 193 days ago | hide | past | pdf | discuss on HN

In plain words: Any kind of random corruption can break down discrete data, and the undoing is learned from noisy snapshots instead of the whole noise trail. It trains faster and generates better than other diffusion methods, and first beats word-by-word models at this scale.

Abstract

We introduce Generalized Discrete Diffusion from Snapshots (GDDS), a unified framework for discrete diffusion modeling that supports arbitrary noising processes over large discrete state spaces. Our formulation encompasses all existing discrete diffusion approaches, while allowing significantly greater flexibility in the choice of corruption dynamics. The forward noising process relies on uniformization and enables fast arbitrary corruption. For the reverse process, we derive a simple evidence lower bound (ELBO) based on snapshot latents, instead of the entire noising path, that allows efficient training of standard generative modeling architectures with clear probabilistic interpretation. Our experiments on large-vocabulary discrete generation tasks suggest that the proposed framework outperforms existing discrete diffusion methods in terms of training efficiency and generation quality, and beats autoregressive models for the first time at this scale. We provide the code along with a blog post on the project page : \href{https://oussamazekri.fr/gdds}{https://oussamazekri.fr/gdds}.

Oussama Zekri, Théo Uscidda, Nicolas Boullé, Anna Korba
arXiv:2603.21342 · stat.ML, cs.AI, cs.CL, cs.LG · submitted Mar 22, 2026
abstract · pdf · html · 37 pages, 6 figures, 13 tables

add comment on HN