Sawtooth – An async, multi-tiered memory framework for LLM agents (github.com)

Sawtooth has introduced a groundbreaking memory framework designed to enhance the performance of Large Language Model (LLM) agents by addressing the limitations of traditional memory systems. Unlike sequential memory frameworks that freeze the application while generating conversation summaries, Sawtooth operates asynchronously, allowing immediate storage of user messages and offloading summarization to a background worker. This innovation not only reduces latency—from an average of 64.15 seconds to just 5.70 seconds—but also preserves 100% recall accuracy by utilizing an immutable ledger to retain critical facts, effectively mitigating hallucination issues often associated with LLMs. The significance of Sawtooth lies in its architectural advancements, which include a multi-tiered buffer system and a focus on determinism in memory management. By enabling faster and more reliable interactions, it promises to enhance user experiences and broaden the applicability of LLMs across various domains. Additionally, its compatibility with local models and cloud APIs, along with comprehensive observability features and auditing trails, positions Sawtooth as a powerful tool for developers aiming to build robust, efficient LLM applications. The research and technical details are encapsulated in a well-documented API, making it accessible for integration into existing frameworks.