Welcome Beam: a 501B open model you can own, customize, and scale (huggingface.co)

The launch of Beam, a cutting-edge open model boasting a staggering 501 billion parameters with 23 billion active per token, marks a significant advancement for the AI/ML community. Tailored for coding and agentic applications, Beam allows teams to deploy a frontier-scale model on their own hardware or in the cloud, fully customizable to specific use cases. Its Rust-based harness is optimized for performance, integrating day-0 support with libraries like transformers, SGLang, and vLLM. Beam is designed for long-context reasoning, handling inputs of up to 1 million tokens, and employs innovative sparse Mixture-of-Experts (MoE) architecture, ensuring it operates efficiently at a fraction of its massive parameter count. Beam's technical highlights include its 52-layer architecture with a mix of sliding-window and global attention mechanisms, advanced context management, and support for various checkpoint formats, including BF16 and FP8. Notably, it emphasizes agentic capabilities, facilitating operations like tool calling with a dedicated XML-style grammar. With prizes of $10,000 being offered for innovative applications of Beam, its launch not only enhances the landscape of large language models but also invites developers to explore its potential in real-world applications and beyond.