Open source alternative to brain by perplexity (github.com)

Perplexity has unveiled "Rudi," an open-source framework that introduces a causal graph memory system for large language models (LLMs). This innovative approach addresses the common inefficiencies in LLM API calls, where the entire conversation context must be resent for each interaction, leading to escalating costs and hitting context limits. Rudi replaces this method with a dependency graph that focuses solely on relevant decision nodes, significantly reducing input token usage—from 828,369 tokens in a standard session to just 152,222 tokens using Rudi over 43 turns, translating to a cost of only $0.34. This development is significant for the AI/ML community as it enhances the efficiency of LLM interactions, ensuring that costs remain consistent regardless of session length. Technical innovations such as "slicing," which minimizes token input by only recalling pertinent information, and "folding," which compresses inactive decision branches while preserving essential rules, are crucial. Rudi's architecture not only maintains decision integrity over extended conversations but also allows for real-time adjustments. This approach empowers developers to utilize LLMs more effectively, with a focus on cost efficiency and retention of critical decision-making rules, making Rudi a transformative tool for building more sophisticated AI applications.