In plain words: They shrank a diffusion-based coding model's numbers to just 2–4 bits after training to cut memory, then tested it on coding problems against its word-by-word counterpart. The diffusion model kept its accuracy better at the lowest bit settings.
Abstract · On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
Auto-regressive Large Language Models (LLMs) achieve strong performance on coding tasks, but incur high memory and inference costs. Diffusion-based language models (d-LLMs) offer bounded inference cost via iterative denoising, but their behavior under post-training quantization (PTQ) has been sparsely explored. We investigate the application and robustness of PTQ techniques, specifically GPTQ and a modified Hessian-Aware Quantization (HAWQ) algorithm, on a diffusion-based coding LLM (CoDA) and observe that these methods applied to CoDA exhibit greater robustness at low bitwidths compared to Qwen3-1.7B, its auto-regressive counterpart, under a standardized evaluation pipeline. We find that in our setup, CoDA exhibits greater robustness at low bitwidths (2-4 bits), with smaller accuracy degradation across HumanEval and MBPP benchmarks. Additionally, mixed-precision configurations derived from HAWQ provide smooth trade-offs across accuracy, latency, and memory. The results suggest that diffusion LLMs may offer advantages for efficient deployment due to more quantization-resilience.
Aarav Gupta, Gururaj Deshpande, Chandreyi Chakraborty
arXiv:2604.20079 · cs.LG, cs.CL · submitted Apr 22, 2026
abstract · pdf · html