about
ExpShield: Safeguarding Web Text from Unauthorized Language Modeling (arxiv.org)
2 points by pona-a on Jan 19, 2025 | hide | past | pdf | discuss on HN

In plain words: Tiny invisible edits to web text, aimed at the few words that spark memorization, stop AI models from storing the text while keeping it readable. In tests, an attack guessing whether text was used in training fell from 0.95 to 0.55.

Abstract · ExpShield: Safeguarding Web Text from Unauthorized Crawling and LLM Exploitation

As large language models increasingly memorize web-scraped training content, they risk exposing copyrighted or private information. Existing protections require compliance from crawlers or model developers, fundamentally limiting their effectiveness. We propose ExpShield, a proactive self-guard that mitigates memorization while maintaining readability via invisible perturbations, and we formulate it as a constrained optimization problem. Due to the lack of an individual-level risk metric for natural text, we first propose instance exploitation, a metric that measures how much training on a specific text increases the chance of guessing that text from a set of candidates-with zero indicating perfect defense. Directly solving the problem is infeasible for defenders without sufficient knowledge, thus we develop two effective proxy solutions: single-level optimization and synthetic perturbation. To enhance the defense, we reveal and verify the memorization trigger hypothesis, which can help to identify key tokens for memorization. Leveraging this insight, we design targeted perturbations that (i) neutralize inherent trigger tokens to reduce memorization and (ii) introduce artificial trigger tokens to misdirect model memorization. Experiments validate our defense across attacks, model scales, and tasks in language and vision-to-language modeling. Even with privacy backdoor, the Membership Inference Attack (MIA) AUC drops from 0.95 to 0.55 under the defense, and the instance exploitation approaches zero. This suggests that compared to the ideal no-misuse scenario, the risk of exposing a text instance remains nearly unchanged despite its inclusion in the training data.

Ruixuan Liu, Toan Tran, Tianhao Wang, Hongsheng Hu, Shuo Wang, Li Xiong
arXiv:2412.21123 · cs.CR · submitted Dec 30, 2024 · updated Dec 16, 2025
abstract · pdf · html · 18 pages

add comment on HN