Why harnesses matter for energy (and token) efficiency (neuralwatt.com)

A recent evaluation by Neuralwatt revealed significant insights into the efficiency of various AI harnesses, focusing on how they handle token caching during sessions. The findings indicate that harnesses can dramatically influence energy consumption: a cache miss can increase energy use by over twofold compared to a cached request. For instance, on the GLM-5.3 model, a mostly uncached request consumes about 2.5 times more energy. The research highlights that common causes of cache misses stem from prompt modifications by the harness itself, particularly when elements of the initial prompt are edited, leading to extensive re-computation. This study is crucial for the AI/ML community as it underscores the relationship between the architecture of AI harnesses and operational efficiency. By examining clients like OpenCode and deepseek-harness, which demonstrate varying cache hit rates, the analysis emphasizes the potential for harness design improvements to optimize energy usage and reduce computational costs. These insights could lead to better strategies for managing prompt caching, thus enhancing overall performance and sustainability in AI applications, particularly in scenarios involving long or complex sessions.