Nvidia-Nemotron-Labs-Dory1.0-4B-NVFP4 (huggingface.co)

NVIDIA has announced the release of Dory1.0-4B-NVFP4, a sophisticated large language model (LLM) designed with a looped, latent-recurrent architecture that incorporates interleaved sliding-window attention and full-attention layers. This model features 4 billion parameters and boasts an impressive context length of up to 256,000 tokens, making it particularly adept for general-purpose reasoning tasks while maintaining a smaller computational footprint. Dory1.0 is available under the OpenMDW License Agreement, enabling both commercial and non-commercial use. This model's architecture allows for enhanced token efficiency; it uses a unique looping mechanism to reduce the number of tokens required for task completion by optimizing computations in latent space rather than generating longer outputs. Additionally, it offers elastic inference capabilities, allowing users to adjust computation budgets without retraining the model, thereby balancing efficiency and accuracy depending on specific tasks. The Dory1.0 is built with NVIDIA's native NVFP4 format, further streamlining its performance while offering support for multiple formats like BF16 and Q5_K_M. This innovation positions NVIDIA at the forefront of AI/ML advancements, providing a powerful tool for developers and researchers aiming for both cost-effectiveness and high performance in language processing applications.