An open-source energy and latency profiler for LLMs: ELANA (arxiv.org)

Researchers have introduced **ELANA** (Energy and Latency Analyzer), an open-source profiling tool designed to evaluate the efficiency of large language models (LLMs). This tool addresses the critical challenges of latency and power consumption associated with deploying LLMs across various hardware platforms, from mobile edge devices to cloud GPU clusters. ELANA provides detailed analysis on aspects such as model size, key-value cache size, and various latency metrics (TTFT, TPOT, TTLT), making it a valuable resource for both model deployment and next-generation LLM development. Significantly, ELANA supports all publicly available models on Hugging Face and features a user-friendly command-line interface, complete with optional energy consumption logging. Its compatibility with Hugging Face APIs and ability to adapt to compressed or low bit-width models positions it as a practical tool for researchers focusing on efficient LLMs or small-scale proof-of-concept studies. By offering this lightweight profiling solution, the development of ELANA not only encourages optimization among developers but also fosters advancements toward more sustainable and responsive AI systems.