about
Neuro-Formal Verification: Agentic Language-Agnostic Formal Program Reasoning (arxiv.org)
3 points by matt_d 18 days ago | hide | past | pdf | discuss on HN

In plain words: An AI agent turns ordinary code into a proof task in a verification-friendly language, which a sound checker settles, with staged steps guarding against proving the wrong thing. On correct and buggy Python programs it reached 92% precision, versus 72% for an unchecked AI judge.

Abstract

Formal verification provides the strongest correctness guarantees for software, and verification-aware languages can produce sound, machine-checked proofs. Recent AI coding agents have sharply lowered the cost of constructing such proofs. Yet few mainstream developers benefit: most use languages without formal-verification support, and formalizing properties and modeling execution environments demand formal-methods expertise. Proof therefore remains reserved for a few notable artifacts, while production software is attested mainly through review and testing. We introduce neuro-formal verification (NFV), which brings this automation to mainstream languages. An AI coding agent formalizes a source-level verification problem into a proof obligation in a verification-aware language, discharged by an established sound verifier aided by agentic proof search. Staged, goal-blind transformations reduce the risk of proving an artifact that does not faithfully represent the source program, property, or environment. Since NFV cannot ensure the soundness of this formalization, it optimizes for empirical accuracy rather than end-to-end soundness, while insisting on machine-checked evidence for every verdict. Experiments with current frontier models on a balanced dataset of correct and buggy Python solutions demonstrate the effectiveness of our approach. NFV with Dafny correctly resolves 57% of all entries, at 92% precision among its verdicts; with a CBMC backend, it produces a counterexample for 63% of the buggy programs at 90% precision. In contrast, an LLM-as-judge baseline achieves only 72% precision while answering every entry without any checkable artifact, and an unstaged agent-verifier combination proves 98% of both the correct and the known-buggy programs, yielding only 50% precision. Together, they confirm that both proofs and staging benefit an AI agent's formal program reasoning.

Shuvendu K. Lahiri
arXiv:2608.21516 · cs.SE, cs.PL · submitted Aug 21, 2026 · updated Sep 14, 2026
abstract · pdf · html · Improve the presentation to clarify some of sentences that did not read well

add comment on HN