Tokdiet a local proxy that cuts LLM token spend ~70% without quality loss (github.com)

Tokdiet has introduced a novel local proxy that effectively reduces token usage by approximately 70% without sacrificing the quality of outputs from large language models (LLMs). This system acts as an intermediary between AI agents and model APIs, optimizing the context sent during requests. By implementing a nuanced approach to context management, Tokdiet allows agents to operate with significantly fewer tokens—going from 5.07 million to 1.46 million token requests—while maintaining competitive performance metrics with an impressive 95-97% quality parity when compared to baseline tests using other models. The significance of Tokdiet lies in its ability to cut costs associated with token usage—crucial for organizations leveraging AI services that often operate on a pay-per-token basis. By incorporating mechanisms such as deduplication, elision, and a strict quality monitoring system, Tokdiet ensures that essential context is preserved, reducing excess without degrading the model's intelligence. With real-time monitoring of both usage and cost savings via an accessible dashboard, Tokdiet not only proves its effectiveness but sets a new standard for context optimization in the AI/ML community.