Amazon EC2 G7e Instances Accelerated by RTX Pro 6000 Blackwell GPUs (aws.amazon.com)

Amazon has launched its new EC2 G7e instances, powered by NVIDIA's RTX PRO 6000 Blackwell Server Edition GPUs, designed to enhance performance for generative AI inference and graphics workloads. These instances offer up to 2.3 times improved inference performance over the previous G6e instances, thanks to advancements like double the GPU memory and increased memory bandwidth. This enables users to run medium-sized AI models with up to 70 billion parameters in FP8 precision on a single GPU. The G7e instances also support NVIDIA GPUDirect P2P technology, which minimizes latency and maximizes performance for multi-GPU setups, allowing for efficient large model inference across up to eight GPUs and a total of 768 GB of GPU memory. Furthermore, the G7e instances provide quadruple the networking bandwidth compared to their predecessors, facilitating small-scale multi-node workloads and improving data throughput significantly. These enhancements not only streamline operations for AI and ML applications but also expand the potential for complex scientific computing and spatial computing tasks, making it a notable advancement in cloud computing capabilities.