about
Parallel Language Models via Variational Decoding (arxiv.org)
4 points by E-Reverance 28 days ago | hide | past | pdf | discuss on HN

In plain words: Instead of writing one word at a time, this model emits many words in a single step, using helper signals to keep them consistent. A 1.3-billion-parameter version matched or beat the usual one-at-a-time model on math, coding and instructions, running 2.3 to 5.7 times faster.

Abstract

Autoregressive language models emit one token per forward pass. This causes high inference latency, in particular as models are scaled up for quality. In this paper, we introduce Parallel Language Models (PLMs), which generate mutually consistent tokens in one forward pass by routing dependencies through latent auxiliary variables. PLMs are trained using a variational bound that reduces to an autoregressive loss plus a parallel decoding loss. Empirically, PLMs are on a faster and better Pareto frontier than autoregressive and discrete diffusion baselines. Pretrained PLMs achieve near-autoregressive pretraining loss on model scales of 80M to 3B parameters, and speedup follows a power law in model size. When post-trained, our 1.3B model matches or exceeds an identically trained autoregressive model on math, coding, and instruction-following benchmarks, speeding up inference $2.3\text{-}5.7\times$. Together, we show that parallel generation can be trained from a single training objective at autoregressive quality. We will make our code available at https://github.com/mandt-lab/plm.

Felix Draxler, Robin C. Geyer, Daniel Geyfman, Farrin Marouf Sofian, Justus Will, Philipp Hildebrandt, Giosue Migliorini, Prakhar Srivastava, Gagan Sai Trinadh Duggaraju Peddiraju, Kushagra Pandey, Jorge da Silva Gonçalves, Elijah Tavares, et al.
arXiv:2610.26693 · cs.LG, cs.CL · submitted Oct 19, 2026
abstract · pdf · html

add comment on HN
Also discussed: Oct 2026 (1 point, 0 comments)