Known by Their Actions: Fingerprinting LLM Browser Agents via UI Traces (arxiv.org)

Researchers have unveiled a method to fingerprint large language model (LLM) browser agents by analyzing their user interface traces. This study, which explored interactions across 14 different LLMs in various web environments, found that it is possible to identify the specific underlying model of an agent with an impressive accuracy of up to 96% F1 score. This capability poses significant security implications, as it could enable attackers to exploit known vulnerabilities specific to certain LLMs by observing their behavior in tasks such as information retrieval and online shopping. The findings reveal that classifiers trained on the actions of these agents can generalize across differing model sizes and families, allowing for the early identification of an agent's identity during interactions. While introducing randomized timing delays between actions can degrade the classifier's performance, it does not provide lasting protection. The researchers have made their harness and a labeled corpus of agent traces publicly available, promoting further exploration into the dynamics of LLM security and privacy. This work highlights the need for heightened awareness about the vulnerabilities associated with LLM agents as they increasingly interact with web environments on behalf of users.