Full-Pipeline Inference Optimization for MiMo-v2.5 Series (mimo.xiaomi.com)

The MiMo-V2.5 model series, which includes MiMo-V2.5 and MiMo-V2.5-Pro, has introduced significant advancements in full-pipeline inference optimization through the innovative use of Hybrid Sliding Window Attention (Hybrid SWA), sparse MoE activation, and multimodal encoders. Hybrid SWA reduces KVCache storage requirements to about 1/7 of that needed for traditional Full Attention mechanisms, enhancing performance and efficiency in long-context and multimodal tasks. This model smartly intermingles local and global attention to balance the computational demands of long-range dependency reasoning, addressing the typical tension between model power and efficiency. Significantly, the MiMo-V2.5 series is designed to tackle challenges in production environments, such as the complexities of cache management and distributed scheduling. The newly implemented SWA-aware prefix cache and KVCache management systems optimize memory usage and improve access patterns, crucially reducing inference costs, especially in long-sequence scenarios. The development of GCache, a high-performance, multi-tier caching system, further enhances throughput and latency efficiency, making it an essential part of the unified training-inference architecture. These optimizations not only realize the theoretical benefits of Hybrid SWA but also enable more efficient deployment in real-world applications, elevating the capabilities of AI systems in processing multimodal inputs.