about
Known by Their Actions: Fingerprinting LLM Browser Agents via UI Traces (arxiv.org)
3 points by sbulaev 142 days ago | hide | past | pdf | 1 comment on HN

In plain words: A hidden script on a webpage can record an AI browsing agent's clicks and timing to guess which model is driving it. Across 14 models, this identified the model with up to 96% accuracy, and random timing delays only briefly fooled it.

Abstract · Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces

As LLM-based agents increasingly browse the web on users' behalf, a natural question arises: can websites passively identify which underlying model powers an agent? Doing so would represent a significant security risk, enabling targeted attacks tailored to known model vulnerabilities. Across 14 frontier LLMs and four web environments spanning information retrieval and shopping tasks, we show that an agent's actions and interaction timings, captured via a passive JavaScript tracker, are sufficient to identify the underlying model with up to 96\% F1. We formalise this attack surface by demonstrating that classifiers trained on agent actions generalise across model sizes and families. We further show that strong classifiers can be trained from few interaction traces and that agent identity can be inferred early within an episode. Injecting randomised timing delays between actions substantially degrades classifier performance, but does not provide robust protection: a classifier retrained on delayed traces largely recovers performance. We release our harness and a labelled corpus of agent traces \href{https://github.com/KabakaWilliam/known_actions}{here}.

William Lugoloobi, Samuelle Marro, Jabez Magomere, Joss Wright, Chris Russell
arXiv:2605.14786 · cs.CR, cs.AI, cs.HC, cs.LG · submitted May 14, 2026
abstract · pdf · html

add comment on HN

Quite an interesting read. I wonder how well this system would work if you used something other than Midscene.js to fingerprint the models? It was limited in scope on how the models interact with the sites for study's sake, but would be interesting to look at further.