Big GPUs don't need big PCs (www.jeffgeerling.com)

In a surprising experiment, a Raspberry Pi 5 was tested against a modern desktop PC for GPU performance across various scenarios, including media transcoding, rendering, and AI inference. Notably, the Raspberry Pi, which has only one PCIe Gen 3 lane, showcased impressive efficiency and performance—especially when configured with multiple GPUs using an external PCIe switch. Remarkably, one user managed to connect four Nvidia RTX A5000 GPUs to a single Raspberry Pi, achieving performance scores within 2% of a high-end Intel server. This development is significant for the AI/ML community as it illustrates the potential for low-cost, energy-efficient computing solutions that can handle demanding tasks typically reserved for larger, more expensive systems. The Pi demonstrated strong performance, particularly in scenarios with shared memory access, and even excelled in efficiency metrics, consuming significantly less power than traditional setups. While some limitations remain, especially with driver support and the reliance on PCIe bandwidth, the findings imply that smaller-scale hardware could effectively contribute to AI model deployment and other GPU-intensive applications, potentially democratizing access to advanced computing resources.