DeepSeek Elastic Compute (DSec) (arxiv.org)

DeepSeek has unveiled the DeepSeek Elastic Compute (DSec), a groundbreaking sandbox infrastructure aimed at facilitating effective large-scale agentic training with large language models (LLMs). Traditional training requires isolated execution environments that can manage complex, stateful interactions across various tasks. DSec not only meets these demands but also enhances performance by integrating a unified SDK that supports different backend technologies, including container, microVM, and full-VM sandboxes. This flexibility allows for high-density execution across a cluster of approximately 160 nodes, capable of handling around 3 million sandboxes daily and sustaining over 5,000 creations per second. The significance of DSec lies in its innovative design, which combines memory sharing and reclamation, CPU scheduling, and a distributed filesystem to improve efficiency and reduce latency. It is particularly beneficial for reinforcing learning applications, as it decouples stateful executions from GPU-based training, effectively maintaining state while optimizing resource usage. This capability is crucial in mitigating issues such as reward hacking by AI agents, ultimately enhancing the reliability and effectiveness of training processes in the rapidly evolving AI/ML domain.