about
The More It Says, the More You Pay: A Black-Box Audit of Token Inflation in LLM (arxiv.org)
2 points by sbulaev 15 days ago | hide | past | pdf | discuss on HN

In plain words: A dishonest pay-per-token service can quietly stretch answers so users pay for extra words. A simple test nudges it to write longer: if it is already inflating, output barely grows, catching 85.1% of attacks with almost no false alarms.

Abstract · The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services

In pay-per-token LLM services, the more a model says, the more users pay. Dishonest providers can covertly manipulate generation to inflate output tokens while largely preserving task utility. We define such manipulation as a Provider-Side Token Inflation Attack (PTIA) and instantiate five representative attacks at the query, prompt, representation, and model levels of the provider-controlled pipeline. Our experiments show that each attack increases mean output length to more than 10.2x the clean baseline, demonstrating PTIA's financial appeal and feasibility at multiple stages of generation. Yet auditing PTIA from black-box responses is difficult for users. Our key observation is PTIA saturation: an initial attack sharply lengthens output, but further strengthening or composition has much less effect. We trace this saturation to stopping behavior: an initial PTIA sharply lowers the end-of-sequence token probability, whereas further intervention lowers it only marginally. Building on this insight, we design a lightweight single-probe audit that applies a controlled lengthening intervention. Under PTIA, the probe induces far fewer additional tokens than under normal service. The audit requires neither a trusted local reference model nor historical clean responses, and its separately issued original and probed requests resemble ordinary traffic, making evasion difficult. Across four open-weight models, it achieves an average detection rate of 85.1% with false-positive rates below 2%. Across 15 real LLM API services, the audit flags 7 for PTIA-consistent behavior.

Leilei Chen, Lan Zhang, Chen Tang, Pengcheng Sun, Jiewei Lai, Yixiao Huang, Zhaopeng Zhang, Xinpeng Shen
arXiv:2609.20370 · cs.CR · submitted Sep 17, 2026
abstract · pdf · html · 22 pages, 10 figures, 8 tables

add comment on HN
Also discussed: Sep 2026 (2 points, 0 comments)