O-POPE: High-Frequency Pipelined Outer Product based GEMM acceleration (arxiv.org)

A new advancement in hardware architecture, dubbed O-POPE, has been introduced to enhance General Matrix Multiply (GEMM) performance in machine learning workloads. Traditional GEMM accelerators struggle to balance high operating frequency, arithmetic utilization, and buffering overhead, particularly for high-precision floating-point operations critical in accuracy-sensitive tasks like training. O-POPE addresses these challenges by repurposing floating-point unit (FPU) pipeline registers as buffers, achieving a remarkable operating frequency of 1 GHz while maintaining less than 2% of buffer area in a 2048-MACs configuration. This innovation is significant for the AI/ML community as it demonstrates how to maximize resource utilization, achieving an impressive 99.97% FPU utilization rate. The O-POPE architecture not only increases performance by 1.33 times compared to existing solutions but also enhances performance density by 9% and energy efficiency by 8%. These improvements suggest a promising direction for developing future high-efficiency accelerators that can handle extensive floating-point operations in machine learning, underscoring the importance of optimizing hardware alongside evolving software demands.