GPT-5.2, Grok 4.1, and DeepSeek v3.2 compare as Santa agents (veris.ai)

Veris AI has introduced a new benchmarking tool, SantaBench, which evaluates leading LLMs - GPT-5.2, Grok 4.1, and DeepSeek V3.2 - in a whimsical holiday challenge where the AI plays a Santa agent that researches users online and delivers humorous roasts based on their social media presence. This benchmark rigorously tests multiple capabilities, including web search, identity verification, and conversational skills, while isolating results through a simulation engine that ensures real-time AI performance without external third-party noise. The significance of SantaBench lies in its ability to quantify AI performance in terms of tool usage reliability and roast quality, revealing a noteworthy tradeoff: DeepSeek showed superior reliability with a 95% tool usage rate but scored lower on humor, while GPT and Grok excelled in delivering entertaining roasts at a rate of 76% for quality humor. This benchmark not only demonstrates the varied strengths of proprietary versus open-source models but also highlights the importance of balancing technical reliability with engaging conversational AI, paving the way for future advancements in agentic applications.