Air-LLM: Broadcasting AI Weights for Edge LLM Inference via RF Computing (arxiv.org)

Researchers have introduced AIR-LLM, a groundbreaking architecture enabling edge devices to perform large language model (LLM) inference without the need to store or load LLM weights locally. By leveraging radio frequency (RF) broadcasting, a central radio—like a 5G base station—transmits the model weights over the air, allowing edge devices to use these weights on-the-fly for computations. This innovation is particularly significant as it addresses the challenges of memory limitations and energy consumption that have hindered the deployment of LLMs on edge devices. AIR-LLM enhances performance through technologies such as MIMO spatial multiplexing and a unique precoder-postcoder pair designed for energy efficiency. The architecture has demonstrated impressive results, achieving a mere 4% degradation in performance on the WikiText-2 dataset using LLaMA-3.1-8B, while delivering energy savings up to 157.7 times compared to traditional methods. Additionally, when serving multiple users, AIR-LLM significantly reduces airtime, showcasing its scalability and efficiency. This research represents a pivotal step toward making advanced AI models accessible on resource-constrained devices, opening new avenues for AI applications in everyday environments.