about
Specula: Scaling formal specs for autonomous model checking of system code (arxiv.org)
3 points by matt_d 65 days ago | hide | past | pdf | discuss on HN

In plain words: Coding agents write formal descriptions of a program's rules and behavior, then mathematically check every possible run for violations, improving their own descriptions through repeated review. Across 48 open-source system projects this found 249 bugs, including deep ones other techniques miss.

Abstract · Specula: Scaling formal specifications for autonomous model checking of system code

Specula is a push-button agentic system that generates high-quality formal specifications for large, complex system code and uses the specifications for highly effective model checking and bug finding. Specula employs large language model (LLM) based coding agents to autonomously develop TLA+ specifications, including invariants that describe correctness properties of the target system and formal models that describe the system implementation with the right level of abstractions. Specula is fully autonomous and thus eliminates the barrier of applying formal methods to real-world system code (as in traditional human-centric approaches). Meanwhile, Specula addresses limitations of LLM-driven techniques like reward hacking and hallucinations through self-evolving loops that iteratively improve specification quality by enabling the agents to deepen their understanding of system code and its behaviors. We have used Specula to check 48 open-source system projects; Specula found 249 bugs including many deep bugs that are hard to find by existing approaches. Specula has been used by several companies and is maintained at https://github.com/specula-org/Specula.

Qian Cheng, Saad Mohammad Rafid Pial, Ruize Tang, Yiming Su, Emilie Ma, Finn Hackett, Ivan Beschastnikh, Yu Huang, Tianyin Xu
arXiv:2607.25333 · cs.SE, cs.AI, cs.DC, cs.OS · submitted Jul 28, 2026 · updated Aug 3, 2026
abstract · pdf · html · 17 pages, 11 figures

add comment on HN