A sovereign, open foundation model for German and English (twitter.com)

The release of the Soofi S 30B-A3B pretraining tech report marks a significant advancement in the development of sovereign, open foundation models for both German and English. This Mixture-of-Experts hybrid model, trained on approximately 27 trillion tokens with a deliberate bias towards the German language, has outperformed existing models, including Olmo 3 32B and Apertus 70B, in comparative evaluations. The project emphasizes radical transparency by providing comprehensive data accounting, hyperparameters, training methods, and model checkpoints, all under permissive licenses, which facilitates further scholarly and practical exploration. The significance of Soofi S lies in its unique training environment on Deutsche Telekom’s Industrial AI Cloud in Munich, promoting sovereign AI infrastructure aligned with local regulations and needs. Additionally, the model achieves impressive performance metrics, combining frontier-level capabilities with high throughput for long-context decoding, a critical aspect for applications in natural language processing. Its ability to maintain efficiency as context grows offers promising implications for future AI applications where understanding extended contexts is essential.