Intelligence per Watt: Measuring Intelligence Efficiency of Local AI (arxiv.org)

A new study introduces a metric called Intelligence per Watt (IPW), which measures the efficiency of local AI models in terms of power consumption and accuracy. As demand for large language models (LLMs) surges, current reliance on centralized cloud computing is strained. Recent advancements in smaller local models (with up to 20 billion parameters) demonstrate competitive performance with larger frontier models, particularly when paired with efficient local hardware like the Apple M4 Max. This raises the possibility of redistributing workloads from cloud servers to local devices, potentially alleviating infrastructural pressure. The research evaluates over 20 state-of-the-art local LMs across eight hardware configurations, using one million real-world chat and reasoning queries. Findings reveal that local models successfully interpret 88.7% of queries, with their IPW improving by 5.3 times from 2023 to 2025 due to advancements in algorithms and hardware. Notably, local accelerators achieve at least 1.4 times lower IPW than cloud counterparts for identical models, indicating ample opportunity for local optimization. The study underscores the growing viability of local inference as a sustainable alternative to cloud systems, marking a significant shift in the AI landscape.