AI Systems Performance Engineering (github.com)

A hands-on course titled "AI Systems Performance Engineering," set for release in November 2025 by O'Reilly, will focus on optimizing AI workloads through advanced GPU techniques, distributed training, and inference scaling. The book aims to address the multifaceted demands of modern AI systems, emphasizing not just raw computational power but "goodput-driven" engineering that enhances the performance and reliability of training and inference pipelines. Readers can expect practical insights on diagnosing bottlenecks, optimizing memory and bandwidth, and employing compiler tools to develop high-impact GPU kernels. For the AI/ML community, this resource is significant as it provides an empirical approach to performance engineering, integrating case studies and thousands of lines of example code in PyTorch and CUDA C++ tailored for NVIDIA GPUs. Key features include a comprehensive 200-item performance checklist encompassing optimization strategies, reproducibility techniques, and power management. The book targets AI/ML engineers, researchers, and systems engineers, offering strategies for scaling AI systems sustainably and cost-effectively while leveraging the latest profiling and tuning tools.