Kimi K2: Open Agentic Intelligence (arxiv.org)

The introduction of Kimi K2, a cutting-edge Mixture-of-Experts (MoE) large language model, marks a significant advancement in open-source AI. With 32 billion activated parameters and a staggering 1 trillion total parameters, Kimi K2 utilizes a novel optimization technique called MuonClip, which incorporates a QK-clip method to enhance training stability while maximizing token efficiency. This model has been pre-trained on an extensive dataset of 15.5 trillion tokens, enabling it to achieve impressive performance in various benchmarks focused on agentic capabilities, including scores of 66.1 on Tau2-Bench and 76.5 on ACEBench (En), surpassing many existing models. Kimi K2's significance lies not only in its technical prowess but also in its applications within the AI/ML community. Its strengths in coding, mathematics, and reasoning tasks make it a versatile tool, especially in software engineering. The implementation of a multi-stage post-training process, which includes reinforcement learning from real and synthetic environments, further enhances its capabilities. By releasing the model checkpoints, the developers aim to foster future research and practical applications of agentic intelligence, which could lead to more autonomous and efficient AI systems across various domains.