Kolibri – Tech Report [pdf] (aleph-alpha.com)

AlephAlpha has introduced Kolibri, a cutting-edge English-German mixture-of-experts transformer model boasting 78.1 billion parameters, with 3.46 billion active per token. Released under the Apache 2.0 license, Kolibri is designed for sovereign and specialized deployment, particularly in regulated domains like public administration, industry, and aerospace. With over 20% of its training data sourced from curated German datasets, Kolibri combines supervised fine-tuning with reinforcement learning across 1.2 million tasks, making it particularly effective for reasoning, instruction following, and code-related applications. This model is notable for its position on the Pareto frontier, optimizing quality against serving costs in both German and English contexts. The significance of Kolibri lies in its alignment with European values and regulations, catering specifically to organizations handling sensitive data. Its development pipeline considers compliance with the EU AI Act and the GDPR, filtering out harmful content while ensuring user data privacy. Additionally, Kolibri can process contexts of up to 1 million tokens, allowing users to balance reasoning effort with operational costs and latency. Overall, Kolibri represents a significant step for European AI development, facilitating local deployment and autonomy while providing robust performance across critical fields.