DeepSeek v4.1 Flash avg 102 tps on 4x RTX6000 pro max-q, 2.1x up from v4-flash (forum.level1techs.com)

DeepSeek has announced the release of version 4.1 of their inference framework, showcasing remarkable performance by achieving an average of 102 transactions per second (TPS) on a workstation outfitted with four NVIDIA Blackwell RTX6000 Pro Max-Q GPUs. This represents a significant 2.1x increase in efficiency compared to the previous version, V4-Flash. The setup, which also includes a 64-core AMD Ryzen Threadripper PRO 7985WX processor and 512GB of RAM, highlights the evolving capabilities of local machine learning inference systems and is geared towards users engaged in advanced machine learning experiments and local inference tasks. The implications for the AI/ML community are noteworthy, as this build not only elevates throughput for large language model (LLM) inference but also presents an accessible reference for developers looking to optimize their own GPU compute environments. The dual-boot capability with Ubuntu and Windows enhances its versatility, catering to various ML workflows. Additionally, users may need to consider potential limitations regarding model support with new architectures, as labs begin to develop for the latest NVIDIA Hopper and Blackwell hardware. The combination of advanced hardware and framework enhancements marks a significant step forward for those working in AI, making powerful, high-performing systems more feasible for a broader range of users.