Lightweight Transformer Architectures for Edge Devices in Real-Time Applications (arxiv.org)

A recent survey highlights advancements in lightweight transformer architectures tailored for resource-constrained edge devices, crucial for enabling real-time AI applications. This comprehensive analysis evaluates various techniques such as model compression, quantization, pruning, and knowledge distillation, and includes performance benchmarks of popular models like MobileBERT and TinyBERT across standard datasets. The findings indicate that these lightweight transformers can maintain 75-96% accuracy while significantly reducing model size and latency—by factors of 4-10x and 3-9x, respectively—making deployment feasible on devices consuming just 2-5W of power. The significance of this work lies in its implications for the broader AI/ML community, particularly in optimizing AI for mobile and edge computing environments. The survey identifies effective strategies like sparse attention mechanisms and mixed-precision quantization, along with a six-step deployment pipeline that achieves substantial model size reduction with minimal accuracy loss. By revealing hardware utilization patterns, optimal parameter counts for efficiency, and energy profiling across various edge platforms, this research provides a roadmap for researchers and developers aiming to deploy AI solutions that are both efficient and effective in real-time applications.