about
Provably unmasking malicious behavior through execution traces (arxiv.org)
46 points by PaulHoule 256 days ago | hide | past | pdf | 5 comments on HN

In plain words: Instead of running suspicious code, the checker asks the model to predict what the code would do across many equivalent rewrites and flags inconsistent answers as possible backdoors. Theory shows no training can fool this check, but early tests produced too many false alarms.

Abstract · The Double Life of Code World Models: Provably Unmasking Malicious Behavior Through Execution Traces

Large language models (LLMs) increasingly generate code with minimal human oversight, raising critical concerns about backdoor injection and malicious behavior. We present Cross-Trace Verification Protocol (CTVP), a novel AI control framework that verifies untrusted code-generating models through semantic orbit analysis. Rather than directly executing potentially malicious code, CTVP leverages the model's own predictions of execution traces across semantically equivalent program transformations. By analyzing consistency patterns in these predicted traces, we detect behavioral anomalies indicative of backdoors. Our approach introduces the Adversarial Robustness Quotient (ARQ), which quantifies the computational cost of verification relative to baseline generation, demonstrating exponential growth with orbit size. Theoretical analysis establishes information-theoretic bounds showing non-gamifiability - adversaries cannot improve through training due to fundamental space complexity constraints. This work demonstrates that semantic orbit analysis provides a theoretically grounded approach to AI control for code generation tasks, though practical deployment requires addressing the high false positive rates observed in initial evaluations.

Subramanyam Sahoo
arXiv:2512.13821 · cs.LG · submitted Dec 15, 2025 · updated Feb 5, 2026
abstract · pdf · html · 13 Pages, A Preprint

add comment on HN

Based on Table 1: This method is actually worse than generating a random number (0-100% independent of the program) and testing if it is less than 98.8%. That would achieve a better detection rate without increasing the false positive rate.

It doesn't seem worth it to try to follow the math to see if there is something interesting.

"'Forbidden' AI Technique" (Computerphile)

https://www.youtube.com/watch?v=Xx4Tpsk_fnM

"The Hard Problem of Controlling Powerful AI Systems" (Computerphile)

https://www.youtube.com/watch?v=JAcwtV_bFp4

Attempting to guide statistical salience of LLM reasoning model procedures, usually just created an evasive interface facade in the output. =3

Interesting direction but the 98.8% FPR in Table 1 seems like a dealbreaker. Anyone understand what's going on with the contradictory results between the text and tables?
> Empirically, CTVP attains very good detection rates with reliable false positives

A novel use of the word "reliable"? Jokes aside, either they mean the FPR as the opposite of what you'd expect, the table is not representative of their approach, or they're just... really optimistic?

  >  Anyone understand what's going on with the contradictory results between the text and tables?
Well Figure 1 would also disagree. It shows a FPR of 47.5%.

From Sec 3, end of second to last paragraph

  | The protocol is deterministic given fixed RNG seeds, caches model outputs
by program hash, and *bounds false positives via the chosen percentile and gap parameters.*

I believe this is a choice, though I think it is suspect that the FPR is pushed this high to get the TP results.

Disclaimer: I only gave this a very cursory skim so don't rely on me too much