Primus Projection: Estimate Memory and Performance Before You Train (rocm.blogs.amd.com)

The Primus Projection tool has been announced to enhance the planning of large-scale distributed training for deep learning models, specifically large language models (LLMs) using AMD Instinct™ GPUs. This innovative tool allows researchers and engineers to estimate memory requirements and performance metrics before conducting expensive training runs, effectively answering critical questions like “Will it fit?” and “How fast will it be?”. By utilizing analytical modeling instead of relying on traditional trial-and-error methods, Primus Projection significantly reduces wasted computational resources and time, a major concern given the costly nature of GPU hours in training large-scale models. The tool operates in two primary modes: memory estimation and performance projection. The memory estimation utilizes a hierarchical approach to predict per-GPU memory consumption based on the model's architecture and parallelism configurations, while the performance projection benchmarks layers on a configurable number of GPUs and uses communication models to project performance across extensive node setups. The hybrid model adapts to the available hardware, performing CPU-based simulations when GPUs are limited, thus optimizing the training setup even during resource constraints. This dual approach promises improved efficiency and effectiveness in managing the increasingly complex training environments in the AI/ML community.