EdgeDAE: Acceleration of Diffusion Action Experts for Real-Time Physical AI (arxiv.org)

A groundbreaking advancement in edge computing for AI has been announced with the introduction of EdgeDAE, a heterogeneous FPGA-GPU system designed to optimize the execution of Diffusion Action Experts (DAE) in real-time physical AI applications. Traditional edge GPU platforms struggle with the iterative denoising process required by DAEs due to memory-bound performance issues, as they frequently load parameters from DRAM, leading to inefficient use of computational resources. EdgeDAE addresses this challenge by offloading vision-transformer workloads to the GPU while utilizing FPGA for DAE inference, significantly reducing memory bottlenecks through the implementation of on-chip parameter storage. The significance of EdgeDAE for the AI/ML community lies in its potential to dramatically enhance inference speed and energy efficiency. The new architecture boasts a 52.5% reduction in end-to-end inference latency for Octo-Small models and a 38.7% reduction for Octo-Base models, all while achieving throughput rates up to 2.10 times higher than traditional edge GPU baselines. Furthermore, compared to high-end consumer GPUs like the RTX 4090, EdgeDAE delivers nearly 17 times greater energy efficiency, marking a substantial leap forward in the performance and sustainability of AI systems deployed in real-time, edge-based environments.