The State of the Open Frontier (maximelabonne.substack.com)

In a recent update on the landscape of AI models, the report titled "The State of the Open Frontier" highlights the emergence of Kimi K3 as a leading open model, surpassing previous versions and closing the performance gap with prominent closed models like Anthropic’s Claude and OpenAI’s GPT. Despite closed models often delivering superior performance, the significance of Kimi K3 lies in its affordability and user control, allowing organizations to host their own models without dependency on external APIs, which are vulnerable to deprecation or restrictions. Kimi K3’s aggressive design, with 2.8 trillion parameters and a novel attention mechanism, positions it as a strong competitor, albeit with known issues such as a higher hallucination rate. The report also reveals that six of the seven top open models are from Chinese labs, revealing a significant divide in model performance and architecture. The development of Sparse Mixture of Experts (MoE) has become standard, but there is a lack of consensus on the best attention design. The adoption of on-policy distillation as a new post-training standard demonstrates a shift in approach toward enhancing model performance. Open models are becoming increasingly competitive, yet closed models still hold an edge in complex coding tasks, indicating ongoing challenges within the AI/ML community as they strive for both innovation and reliability.