Nvidia releases 8B model with learned 8x KV cache compression (huggingface.co)

Nvidia has announced the release of the Qwen-3-8B-DMS-8x, an advanced AI model that utilizes Dynamic Memory Sparsification (DMS) for optimizing key-value (KV) cache memory during inference. This model achieves an impressive 8x compression ratio, enhancing both throughput and latency for long-context reasoning tasks. DMS adapts its eviction strategies to vary between a window of the most recent 512 tokens and full attention, providing a more efficient framework for managing memory during model operation. This model is intended strictly for research and educational purposes under the NVIDIA License. The significance of the Qwen-3-8B-DMS-8x in the AI/ML community lies in its potential to improve resource efficiency in large language models (LLMs). With 8.2 billion parameters and a native context length that can extend to over 131,000 tokens, it promises to streamline operations on NVIDIA's hardware platforms, such as the Ampere and Hopper architectures. Developers interested in leveraging its capabilities can access its weights and inference code through Hugging Face, marking a step forward in practical applications for AI that can efficiently manage larger contexts and complex reasoning tasks while reducing the computational overhead typically associated with LLMs.