Google's new AI scheduler optimizes resource usage in large cloud data centers (research.google)

Google announced LAVA, a suite of lifetime-aware scheduling techniques that uses continuous ML reprediction to dramatically improve VM packing in large data centers. Instead of a single point estimate at VM creation, the system predicts a probability distribution over VM lifetimes (inspired by survival analysis) and continuously updates expected remaining lifetime as VMs run. Three algorithms leverage these repredictions: NILAS (a non-invasive scoring function integrated into Borg that groups VMs with similar exit times), LAVA (intentionally fills long-lived hosts with many short-lived VMs to avoid prolonging host lifespans), and LARS (orders migrations by predicted remaining lifetime to cut unnecessary moves). To avoid a circular dependency and keep latency low, Google compiled the model into the Borg scheduler binary (median inference latency 9 µs) and added a host lifetime score cache to limit prediction volume. The results are tangible: production NILAS deployments increased empty hosts by 2.3–9.2 percentage points and reduced CPU/memory stranding by ~3%/2%; simulations project LAVA to add ≈0.4 pp and LARS to cut migrations by ~4.5%. Beyond immediate capacity and energy gains, this work shows that co-designing probabilistic lifetime models, continuous reprediction, and scheduler-level integration can deliver robust, low-latency ML-driven infrastructure improvements—and is broadly applicable to other resource-allocation and defragmentation problems.