Toolgz – cut LLM tool-definition tokens ~80% without hurting accuracy (github.com)

A new tool called Toolgz has been introduced to optimize large language model (LLM) interactions by significantly reducing tool-definition token usage—by approximately 80%—without sacrificing accuracy. Typically, LLMs spend an extensive amount of context tokens on tool definitions, with realistic setups consuming around 20,000 to 40,000 tokens. Toolgz addresses this inefficiency by compressing tool definitions while enhancing model reasoning capabilities. By connecting to a few model-configurable protocol (MCP) servers, which ship multiple tools formatted as JSON Schemas, this tool compresses the context needed for LLMs to correctly utilize these tools, thereby improving both latency and cost efficiency. The significance of Toolgz lies in its ability to convert a recall problem into a retrieval problem, allowing models to efficiently look up necessary tool information without excessive context consumption. In extensive tests across various models—including Anthropic and OpenAI—Toolgz demonstrated substantial reductions in token count and variations in operational costs, sometimes achieving latencies faster than traditional setups. With features that allow developers to maintain accurate tool definitions and prevent malformed argument transmissions, Toolgz presents a compelling solution for AI developers looking to optimize resource allocation in LLM applications while maintaining performance.