DeepSeek-v4.1-Exp (huggingface.co)

DeepSeek has launched DeepSeek-V4.1-Flash, a cutting-edge multimodal Mixture-of-Experts (MoE) model featuring a staggering 552 billion backbone parameters and the capability to handle contexts up to one million tokens. This model seamlessly integrates image and text processing and utilizes an innovative Causal Encoder-Decoder architecture. This approach enhances efficiency by activating only a fraction of the model's parameters during different operational phases, thus optimizing performance for complex, input-heavy tasks while significantly reducing resource consumption. Significantly, DeepSeek-V4.1-Flash implements advanced techniques like Compressed Sparse Attention 2 (CSA2) and Single-Pass mHC to streamline attention mechanisms and memory usage. CSA2 enables selective attention modes that adjust based on layer requirements, while efficient cache management techniques reduce the global KV cache footprint to just 890 bytes per token, showcasing a remarkable reduction compared to previous versions. Coupled with its extensive training on a multimodal corpus comprising 45 trillion tokens, this model represents an important step forward in developing AI systems capable of more complex reasoning and multitasking capabilities, solidifying its relevance in the evolving landscape of AI/ML technology.