Run Ray on TPU, Part 1: The foundations (developers.googleblog.com)

Ray has announced that its distributed computing framework now fully supports Google Cloud TPUs as first-class accelerators, allowing developers to scale their Python code written for Ray on GPUs to run seamlessly on TPUs. This integration, introduced in Ray version 2.55, eliminates the earlier experimental stage where users had to manually configure their environments. With official APIs and pre-built images, developers can utilize the existing task-and-actor model and transfer their workloads to TPU slices managed by Google Kubernetes Engine (GKE) with minimal adjustments in their code. The significance of this development lies in the enhanced efficiency it provides for resource allocation and execution for AI/ML applications, notably when utilizing TPUs, which can accelerate workloads significantly compared to traditional CPUs and GPUs. A key technical consideration is the need to manage TPU “slices”—a collection of TPU chips that have a high-speed interconnect among them. Developers must ensure that all workers reside on a single slice to avoid synchronization issues during computations. Ray simplifies this process through its slice placement group API, which reserves entire slices atomically and handles the necessary configurations to optimize task execution on TPUs, enabling users to focus on building their AI solutions rather than wrestling with infrastructure complexities. Part 2 of the series will delve into applying Ray's AI libraries on TPUs for various tasks, including serving and training.