Inference Optimization for MiniMax Sparse Attention (www.together.ai)

Together AI has been named the preferred cloud partner for MiniMax's newly launched state-of-the-art model, M3, which features advanced capabilities like a 1M-token context window, multimodal reasoning, and enhanced coding performance. Together AI will host this open-weights model as a developer endpoint upon its public release. Notably, significant optimizations by Together AI's engineering teams, including a KV-Block-Major sparse attention kernel and a Rust-based preprocessing gateway, have achieved throughput improvements of 81-125% under various concurrency levels. These advancements position Together AI as a leading inference platform capable of efficiently deploying complex models at scale. The MiniMax Sparse Attention (MSA) architecture is a standout feature of M3, addressing attention-computation limitations from earlier models and enabling practical long-context processing by capping the number of tokens each query attends to. This results in over a 9x speedup for pre-filling and a more than 15x enhancement during decoding stages. Additionally, the integration of paged attention optimizes KV cache management, while multimodal preprocessing offloads heavy initial processing tasks, ensuring that the GPU can focus on generation tasks. Together AI's close collaboration with MiniMax ensures that the M3 model remains efficient, scalable, and prepared for real-world applications involving extensive documents, codebases, and visual data.