Kimi K3.1 Pro model weights are out (huggingface.co)

Kimi AI has released the weights for its K3.1 Pro model, hailed as its most capable multimodal agentic model yet, categorized within the 3T-class. Building on the previously established Kimi K3, K3.1 Pro underwent extensive post-training utilizing large-scale joint reinforcement learning across a broad spectrum of domains. With a remarkable context window of 1 million tokens, it offers advancements in areas such as coding, 3D creation, and financial research, showcasing noteworthy enhancements in factual accuracy and agentic capabilities. The K3.1 Pro features a sophisticated architecture utilizing a mixture-of-experts (MoE) model, comprising a total of 2.8 trillion parameters, with 104 billion activated parameters across 93 layers. Key technical specifications include a unique attention mechanism employing a combination of KDA and Gated DeepSeek Sparse Attention (Gated DSA), providing improved long-context modeling while maintaining efficiency. This model is significant for the AI/ML community, pushing boundaries in multimodal AI performance and efficiency, which can lead to the development of more advanced applications across various domains. Its competitive benchmarks against other leading models like GPT-6 Astra and Claude demonstrate its potential to redefine standards in AI capabilities.