Kimi K3: second only to Fable 5 on AA-Briefcase (artificialanalysis.ai)

Last week, Moonshot AI unveiled Kimi K3, a new 2.8 trillion parameter AI model that scored 57 on the Artificial Analysis Intelligence Index and an Elo of 1543 on the AA-Briefcase benchmark, making it the second highest scorer behind Claude Fable 5. Kimi K3 demonstrates a remarkable improvement of +727 over its predecessor, Kimi K2.6, showcasing strong analytical capabilities but slightly weaker presentation quality. This model's evaluation on AA-Briefcase involves realistic tasks requiring high-quality deliverables like spreadsheets and presentations, marking it as a significant player in agentic knowledge work. Despite its impressive performance, Kimi K3 comes with a high operational cost, averaging $10.57 per task, which is about ten times more expensive than its counterparts. It also requires nearly an hour to complete each task due to its high output and token usage, averaging 83 turns per task. This increase in complexity and cost reflect both the advancements in AI capabilities and the ongoing challenges related to efficiency and pricing in the competitive landscape of AI/ML models. As more models reach high performance standards, the price of cutting-edge intelligence is rapidly evolving.