A guide on how to run Nemotron 3 Super 120B Thinking on 2 Nvidia DGX Spark (corti.com)

A new guide has emerged detailing the setup and execution of NVIDIA's Nemotron-3 Super 120B model on two DGX Spark workstations, each equipped with powerful Grace-Blackwell SoC and 128 GB of unified memory. The guide highlights the challenges faced during this process, including issues with environment variable propagation, missing software dependencies, and the importance of using the correct model-specific configurations. Notably, the inclusion of advanced tensor cores allows the model to efficiently handle a massive context of 1 million tokens, thanks to its architecture, which combines LatentMoE and attention layers while minimizing memory overhead. This guidance is significant for the AI/ML community as it provides crucial insights into configuring complex AI model deployments, particularly at the high-performance intersection of deep learning and distributed computing. The technical details reveal emerging best practices for managing compute resources effectively, including nuanced Docker configurations and environment management across nodes. Importantly, the guide serves as a resource for developers and researchers looking to replicate similar setups, ensuring they can overcome common pitfalls in scaling and optimizing large-scale AI models. Access to the full repository further facilitates knowledge sharing in this rapidly evolving field.