Show HN: Memory for LLM apps that cuts input tokens up to 80% (avg 68%) (github.com)

Street AI has introduced a groundbreaking memory layer for large language model (LLM) applications that significantly reduces input token usage by averaging around 68%, achieving savings between 55% and 80% in a 16-turn benchmark. This innovative tool sits between user applications and LLM APIs, storing relevant past interactions as organized signals and automatically decaying outdated information. By retrieving only pertinent context when needed, developers can send shorter prompts to LLMs while maintaining conversational continuity, effectively lowering costs associated with token usage. The significance of this memory system lies in its potential to enhance the efficiency of AI-driven applications, making them more scalable and less expensive to operate. The Street AI memory layer is adaptable, supporting multiple LLM providers through a consistent API that integrates seamlessly with existing architectures. Key technical features include persistent memory storage via SQLite, user-specific memory IDs to prevent data leakage, and the ability to adjust configuration settings for optimizing performance. This flexibility, combined with built-in mechanisms for boosting or demoting memory based on interaction outcomes, paves the way for more intelligent and responsive AI applications that can learn from past interactions over time.