about
Agentic backdoors via pretraining poisoning (arxiv.org)
46 points by gmays 1 hour ago | hide | past | pdf | 11 comments on HN

In plain words: Attackers can hide a trigger in a coding agent by planting crafted text in its pretraining data, making it run chosen commands on your computer. Poisoning 0.1% of the text made the largest model obey 92% of the time while acting normally otherwise.

Abstract · Agentic Backdoors via Pretraining Poisoning

Large language models are pretrained on vast, uncurated corpora scraped from the public web. This creates a security risk from attackers who plant poisoned content in the pretraining data. We show for the first time that such poisoning can plant a remote code execution backdoor in an agent, causing it to execute attacker-chosen commands on the user's machine. To measure the threat end to end, we pretrain Qwen3 models from scratch at sizes 0.6B, 1.7B, and 4B, and then post-train each into a CLI agent using a clean pipeline. Poisoning only 0.1% of the pretraining corpus with our new poison design implants the backdoor: the deployed agent executes the malicious command when the trigger is present while behaving normally on clean inputs. The attack success rate (ASR) increases with model scale, with the 4B model reaching roughly 92% ASR at a 0.1% poison rate and staying positive even when the poisoning rate drops to 0.0005%. Our code is available at https://github.com/safety-research/agentic-backdoor.

Xinyan Hu, Pingbang Hu, Chloe Loughridge, Xuanli He, Nicholas Carlini, Matthew Jagielski, Javier Rando, Yiming Zhang
arXiv:2610.37741 · cs.CR, cs.CL, cs.LG · submitted Oct 27, 2026
abstract · pdf · html

add comment on HN
Also discussed: Nov 2026 (2 points, 1 comment)

Having strong sandboxing and observability around the models is clearly a hard requirement if you want to run these things autonomously.
I've been thinking about this for about a year. In my head I've called it "context bombs", though that may not be super accurate.

Think back to the movie The Manchurian Candidate. Produce highly capable LLM models then give them away for free. Train them with key phrases/codes etc to perform nefarious actions later when those codes are present in some piece of ingested context.

There were a couple of papers written on the topic in 2018 (I think, I'll need to find them again) and there's been surprisingly little more work done on the topic since. At least from what I've seen.

I really love the use of local models over subscription services but personally I would never recommend using open weights models from certain sources in a commercial environment.

I just sort of presumed this was already being done. Because why wouldn't it be?
If you were a malicious actor do you really think this method is going to offer the greatest return on investment?
Nation states aren’t looking for a return on investment rather they have a particular objective they want to achieve, e.g. stuxnet or the pager attack.
Sew a million seeds, see what grows.
If LLMs can edit weights directly to achieve desired ends as recently reported[0], then this class of Agentic Backdoors likely becomes even easier and more prevalent.

[0]https://news.ycombinator.com/item?id=50341174

Downloading models off huggingface feels like the new warez.

Sure, the person who abliterated/edited that model might have given you an amazing unlock for free, or they might have given you a credential stealer.

few places up on front page is a post regarding big model improving a smaller one. Lo and behold, next year big models will search for backdoors in smaller models who will be used to get the bigger models out of work.