Replacing RAG with a cognitive memory stack in Elixir/OTP (0xcc.re)

A new architecture for long-lived AI agents, dubbed "Skynet," aims to revolutionize how reinforcement and memory are managed in conversational AI using Elixir's actor model. What started as a humorous exploration of large language model (LLM) interactions transformed into a sophisticated system to address the “amnesia problem” inherent in LLM applications, whereby models forget earlier context in lengthy conversations. Traditional retrieval-augmented generation (RAG) methods, which assemble fragmented responses based on vector searches, often lead to incoherent outputs. In contrast, Skynet's agents—called Souls—leverage a layered cognitive memory stack inspired by neuroscience to maintain consistent state across interactions, improving coherence and efficiency. The structural design of Skynet incorporates a three-tiered memory system that mirrors cognitive functions in biological organisms: short-term, medium-term, and long-term memory. This allows agents to recall relevant information efficiently while minimizing latency through a stable context between interactions. Key features include a decay mechanism for outdated memories, similarity-based association tracking, and a unique "surprise" metric that triggers immediate memory updates in response to novel events. By integrating these neural-inspired strategies, the initiative not only enhances the robustness of AI memory systems but also streamlines interaction costs, making it a significant advancement for the AI/ML community focused on creating more intelligent, adaptive agents.