Grounding AI shopping agents using personas learned from raw clickstream data (arxiv.org)

A new framework called SimPersona has been introduced to enhance AI shopping agents by learning discrete buyer personas directly from raw clickstream data. Traditional methods of personalization often result in rigid averages that fail to reflect the diverse behaviors of real shoppers. SimPersona leverages a behavior-aware VQ-VAE to capture the statistical nuances of buyer behavior and merchant-specific populations, allowing LLM-based web agents to utilize these personas as compact tokens for more accurate representation during shopping tasks. This innovation is significant for the AI/ML community as it enables the creation of more nuanced and adaptable shopping agents that can align closely with actual buyer behavior. Evaluated on over 8 million users across 42 real storefronts, SimPersona demonstrated a remarkable 78% alignment in conversion rates with real buyers, while also providing interpretable variation among different buyer types. Furthermore, the framework includes an open-source data pipeline to transform raw e-commerce logs into usable buyer representations, enhancing the scalability and efficiency of training agent models without complex re-engineering.