about
Token-Efficient Logical Supervision for Mathematical Reasoning (arxiv.org)
1 point by wslh 21 days ago | hide | past | pdf | discuss on HN

In plain words: The model is trained only on the first planning step—picking which variables and operation to use—forcing it to work out the logic itself. It beat full-solution training on familiar and new problems, scoring 4.6% higher on the new ones and training faster.

Abstract · From Implicit to Explicit: Token-Efficient Logical Supervision for Mathematical Reasoning in LLMs

Recent studies reveal that large language models (LLMs) exhibit limited logical reasoning abilities in mathematical problem-solving, instead often relying on pattern-matching and memorization. We systematically analyze this limitation, focusing on logical relationship understanding, which is a core capability underlying genuine logical reasoning, and reveal that errors related to this capability account for over 90\% of incorrect predictions, with Chain-of-Thought Supervised Fine-Tuning (CoT-SFT) failing to substantially reduce these errors. To address this bottleneck, we propose First-Step Logical Reasoning (FSLR), a lightweight training framework targeting logical relationship understanding. Our key insight is that the first planning step-identifying which variables to use and which operation to apply-encourages the model to derive logical relationships directly from the problem statement. By training models on this isolated step, FSLR provides explicit supervision for logical relationship understanding, unlike CoT-SFT which implicitly embeds such relationships within complete solution trajectories. Extensive experiments across multiple models and datasets demonstrate that FSLR consistently outperforms CoT-SFT under both in-distribution and out-of-distribution settings, with average improvements of 3.2\% and 4.6\%, respectively. Moreover, FSLR achieves 4-6x faster training and reduces training token consumption by over 80\%.

Shaojie Wang, Liang Zhang
arXiv:2601.03682 · cs.CL, cs.AI · submitted Jan 7, 2026 · updated Apr 18, 2026
abstract · pdf · html

add comment on HN