LoopLynx: A Scalable Dataflow Architecture for Efficient LLM Inference (arxiv.org)

Researchers have introduced LoopLynx, a novel dataflow architecture designed to enhance the efficiency of large language model (LLM) inference, particularly when leveraging Field-Programmable Gate Arrays (FPGAs). This architecture merges spatial and temporal design strategies, enabling computationally intensive tasks to be processed as large dataflow kernels. By organizing these kernels for reuse, LoopLynx maximizes throughput akin to spatial architectures while also addressing resource limitations through a multi-FPGA setup. This configuration effectively conceals data transfer delays, ensuring optimal utilization of distributed accelerators. The significance of LoopLynx lies in its potential to greatly improve inference speed and energy efficiency for LLMs. Evaluation against the GPT-2 model demonstrates that LoopLynx not only matches the performance of leading single FPGA accelerations but also offers remarkable advantages over conventional chips, such as the Nvidia A100, achieving a 2.52x reduction in inference latency with only 48.1% of the energy expenditure. This advancement could empower the AI/ML community by facilitating the scaling of complex models and fostering model parallelism, thus meeting the growing demands for high-performance computational resources in machine learning tasks.