Mellum2 Goes Open Source: A Fast Model for AI Workflows (blog.jetbrains.com)

Mellum2, a new 12 billion parameter AI model, has been announced as open-source and is specifically designed for enhanced AI workflows in software engineering. Its unique Mixture-of-Experts (MoE) architecture allows for only 2.5 billion parameters to be active per token, significantly reducing compute costs while delivering high throughput and low latency. This makes it an ideal choice for routing, Q&A, and sub-agent tasks in production environments, enabling efficient AI task execution without the need for larger, more resource-intensive models. The significance of Mellum2 lies in its specialized focus on natural language and code, setting it apart from multimodal models while streamlining AI operations in software development. Its capabilities encompass key use cases such as orchestrating AI workloads, building low-latency retrieval-augmented generation (RAG) pipelines, and facilitating private local deployments. With its focus on performance efficiency and practical application, Mellum2 embodies the idea of "focal models" as critical components in the next generation of AI software tooling, aiming to reduce latency and cost in increasingly complex AI systems.