Show HN: Running over 80M tokens in one agent session with no compaction (github.com)

A new implementation called pi-cwl has been introduced, which enhances the capacity of long-horizon LLM agents to manage context without the need for traditional compaction methods. This approach is outlined in the paper “Beyond Compaction: Structured Context Eviction for Long-Horizon Agents” by Andrew Semenov and Svyatoslav Dorofeev. Unlike previous methods that pause performance to summarize data—often leading to loss of essential information or context—the CWL framework organizes the session's history into a structured graph. This allows the agent to efficiently manage extensive data (up to 80 million tokens) without compromising task accuracy or introducing hallucinations. The significance of CWL for the AI/ML community lies in its ability to maintain context integrity through a deterministic eviction policy that prioritizes preserving critical dependencies during data striping. It achieves a remarkable balance of efficiency, completing 89 sequential tasks while maintaining task accuracy rates comparable to isolated sessions. Additionally, CWL promises a reduction in inference costs by 20-70%, making it a compelling solution for developers working with extensive conversational or operational contexts in LLM applications. This innovation could pave the way for further advancements in developing more powerful, context-aware AI agents.