Huawei OpenPangu-2.0-Flash, 512K ctx, 92A6B, Weights+inference code+training ops (twitter.com)

Huawei has announced the open-sourcing of OpenPangu-2.0-Flash, a significant milestone for the AI/ML community. This release includes two language models with 512K context capabilities: the Flash model, featuring 92 billion total parameters and 6 billion active parameters, and the upcoming Pro model, which boasts 505 billion total parameters with 18 billion active, set to be released in July. Along with the models, Huawei has made available the weights, inference code, and training operations, enhancing the accessibility of advanced AI tools for developers and researchers. The significance of this announcement lies in the comprehensive open-source approach that Huawei is adopting, which is expected to accelerate innovation and collaboration in the field of large language models. By providing substantial technical resources, Huawei enables the AI/ML community to experiment with and build upon these models, potentially leading to advancements in understanding and utilization of AI. Furthermore, with plans for additional open-source components later this year, Huawei is positioning itself as a key player in fostering an open ecosystem for AI development.