Nvidia/Nvidia-Nemotron-3.5-Bolt-74B-A9B-24GB (huggingface.co)

NVIDIA has unveiled the Nemotron 3.5 Bolt 74B-A9B 24GB, a revolutionary vision-language model (VLM) engineered for executing everyday agentic tasks locally. With a total of 74 billion parameters (9 billion active), this model employs a 2-bit quantization to efficiently fit within 24GB of memory, making it faster than traditional dense models while maintaining enhanced intelligence compared to many mixture-of-experts (MoE) models of similar memory usage. It supports a diverse range of inputs including text, images, and video, and is designed for optimal deployment in environments with limited resources. The significance of the Nemotron 3.5 lies in its ability to balance speed and intelligence, opening new avenues for local AI deployments without the need for cloud-based processing. With the capability to handle up to 256,000 tokens, it integrates a hybrid architecture combining Mamba-2 transformer and Latent MoE with a state-of-the-art vision encoder. This model is poised to enhance agentic tasks such as tool calling and web browsing across multiple languages, thereby broadening accessibility in AI applications. Moreover, it comes under an open license, encouraging developers to build upon its innovative architecture for various applications in AI and machine learning.