about
Answer First, Reason Later: Commitment Order in Diffusion LLMs (arxiv.org)
3 points by sbulaev 56 days ago | hide | past | pdf | discuss on HN

In plain words: Diffusion language models fill in many words at once and can lock in the final answer before the reasoning that supports it. Reserving the answer slot until after the reasoning is written beat delaying the reasoning by the same amount on three models tested.

Abstract · Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Models

Masked diffusion language models revise many masked output positions in parallel. We call a token committed once it becomes visible and is never masked again, and call a response answer-first when the final answer commits before the reasoning printed ahead of it. On 1,069 GSM8K test questions, an explicit step-by-step instruction increases the accuracy difference between unrestricted decoding and a decoder that permits commitment only near the left-most unresolved position; unrestricted decoding also produces more answer-first trajectories. On MATH-500, the two LLaDA models spend most of a short output canvas on reasoning that commits after the answer, and the benefit of frontier gating decreases as that postanswer writing disappears. Dream-7B has little post-answer writing and follows a different accuracy pattern. A controlled four-option task reserves a one-token answer position before generation. Delaying that position outperforms an equally timed reasoning-token delay on LLaDA-8B, LLaDA-1.5, and Dream-7B. The raw difference is largest on Dream, whose free accuracy on the controlled task is lower. Answers commit much earlier under the reserved-position interface than in ordinary free-form generation, which limits how far the intervention result can be generalized. Commitment order affects the context used to complete a response and the allocation of a finite output canvas.

Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Hwiyeong Lee, Taesup Kim
arXiv:2608.05687 · cs.CL, cs.AI · submitted Aug 6, 2026 · updated Aug 24, 2026
abstract · pdf · html

add comment on HN