about
Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelines (arxiv.org)
1 point by sbulaev 16 days ago | hide | past | pdf | discuss on HN

In plain words: Tool-using AIs mix instructions from several sources in one memory, so attackers split a harmful command into harmless pieces the model reassembles. Models blocking it in one source leaked data up to 100% of the time when split across two; security tools missed it.

Abstract

The Model Context Protocol (MCP) enables LLMs to invoke external tools, but every tool interaction exposes the model to attacker-controlled text through multiple input channels (tool descriptions, tool results, sampling messages) that share a single context window without privilege separation. In this paper, we present a framework to measure the trust profile of an arbitrary LLM based on a variety of payload framings sent through different channels. Following this assessment, we devise cross-channel fragmentation attacks that distribute seemingly benign payloads across two or three channels; no individual channel carries a complete injection, yet the LLM compiles the fragments into credential exfiltration. We evaluated our attacks across 12 frontier models, three production clients, and six payloads, totalling over 15,000 trials. Our evaluation reveals that cross-channel attacks are an unexplored attack surface: models that fully resist single-channel injection (0% compliance) exfiltrate sensitive data at up to 100% under two-channel fragmentation (e.g., GPT-4o, Llama 70B, Composer 2, Haiku 4.5). We further demonstrate value-aligned exploitation, where a tool's stated purpose requires the data the attacker targets, and a sampling system prompt override that injects persistent instructions via VS Code's MCP implementation. Finally, we evaluated our attacks against seven third-party MCP security tools and three prompt-based defenses. All tools failed to detect fragmented payloads, and prompt defenses proved model-specific rather than universal.

Murali Ediga, Sudipta Chattopadhyay
arXiv:2609.18217 · cs.CR · submitted Sep 16, 2026
abstract · pdf · html

add comment on HN