DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles (www.lmsys.org)

DeepSeek-V4 has officially launched with Day-0 support for both inference and reinforcement learning (RL) training using the SGLang and Miles open-source stack. This new model showcases significant advancements in AI/ML, introducing a hybrid sparse-attention architecture and innovative optimization techniques that facilitate efficient processing of large-scale contexts—up to 1M tokens. The architecture employs manifold-constrained hyper-connections (mHC) to enhance gradient flow, alongside native FP4 expert weights designed for optimized performance on cutting-edge hardware platforms. The release is particularly noteworthy for its introduction of ShadowRadix, a novel prefix caching mechanism that streamlines hybrid attention management, enabling it to handle complex token requests without compromising speed or efficiency. This system significantly improves throughput while maintaining the integrity of the computation, even during high-demand scenarios such as speculative decoding. Additionally, DeepSeek-V4 integrates several kernel optimizations, including the Flash Compressor and Lightning TopK, which drastically reduce processing latency and memory overhead. Overall, these technical advancements position DeepSeek-V4 as a powerful tool for enhancing the capabilities of AI models, particularly in real-time applications requiring swift and accurate inference.