about
Agentic Code Reasoning (arxiv.org)
3 points by gmays 212 days ago | hide | past | pdf | discuss on HN

In plain words: A structured prompting style makes coding agents list explicit assumptions, walk each execution path, and justify every conclusion, so they can judge code by reading it rather than running it. It beat free-form reasoning on every task, raising patch-equivalence accuracy from 78% to 88%.

Abstract

Can LLM agents explore codebases and reason about code semantics without executing the code? We study this capability, which we call agentic code reasoning, and introduce semi-formal reasoning: a structured prompting methodology that requires agents to construct explicit premises, trace execution paths, and derive formal conclusions. Unlike unstructured chain-of-thought, semi-formal reasoning acts as a certificate: the agent cannot skip cases or make unsupported claims. We evaluate across three tasks (patch equivalence verification, fault localization, and code question answering) and show that semi-formal reasoning consistently improves accuracy on all of them. For patch equivalence, accuracy improves from 78% to 88% on curated examples and reaches 93% on real-world agent-generated patches, approaching the reliability needed for execution-free RL reward signals. For code question answering on RubberDuckBench Mohammad et al. (2026), semi-formal reasoning achieves 87% accuracy. For fault localization on Defects4J Just et al. (2014), semi-formal reasoning improves Top-5 accuracy by 5 percentage points over standard reasoning. These results demonstrate that structured agentic reasoning enables meaningful semantic code analysis without execution, opening practical applications in RL training pipelines, code review, and static program analysis.

Shubham Ugare, Satish Chandra
arXiv:2603.01896 · cs.SE, cs.AI, cs.PL · submitted Mar 2, 2026 · updated Mar 4, 2026
abstract · pdf · html

add comment on HN
Also discussed: Mar 2026 (1 point, 0 comments)