Reduce LLM token costs 40-60% for structured data (github.com)

A new Python library called Token Optimized Object Notation (TOON) has been announced, designed to reduce token usage by 40-60% when sending structured data to large language models (LLMs). Traditional methods, like JSON arrays, require repetitive attribute names for each record, leading to increased costs, especially at enterprise scale. TOON optimally separates the schema from the data, allowing attribute names to be declared only once, significantly lowering token consumption when processing extensive datasets. This development is significant for the AI/ML community as it addresses the growing concern about token costs in the context of LLM utilization. By employing distinct syntax—like using pipe characters to delimit values and a schema declaration line—TOON enhances the efficiency of data transmission while facilitating simpler integration with LLMs. With TOON, enterprise users can process larger volumes of data at reduced costs, making it particularly valuable for batch processing scenarios and analytical workloads where efficiency is paramount.