No Idle GPUs: Managing Research Compute at Runway (runwayml.com)

Runway has achieved a notable increase in GPU utilization—over 20 percentage points—by implementing Kueue as a Kubernetes admission controller. This system allows the company to allocate reserved quotas for critical workloads while creating a shared queue that effectively borrows idle GPU capacity and preempts it when necessary. By balancing guaranteed access to GPUs for essential training runs with high utilization rates, Runway mitigates the costs associated with idle resources while supporting the varied needs of its teams, including multi-week pretraining jobs and real-time inference tasks. The significance of this development lies in its advanced approach to resource management in high-demand environments. Kueue integrates seamlessly into existing Kubernetes systems without introducing additional complexity, facilitating features like gang scheduling and workload prioritization. By addressing common challenges in multi-tenant GPU clusters—such as the conflict between guaranteed capacity and efficient utilization—Kueue enhances operational efficiency and research velocity. This structured resource allocation strategy not only optimizes GPU usage but also reflects a broader trend in the AI/ML community where effective orchestration of compute resources is critical for advancing experimentation and innovation.