Building AI Memory at 10M+ Nodes: Architecture, Failures, and Lessons (blog.getcore.me)

A recent announcement from CORE reveals significant challenges and advancements in building an AI memory system capable of handling over 10 million nodes. The team learned that static embeddings and conventional vector databases struggle with temporal and contextual queries, resulting in inconsistencies and inaccurate responses. To address this, they developed a memory architecture that integrates a knowledge graph with reified triples, enabling the tracking of factual changes over time. This innovative approach allows for nuanced queries, such as determining what was true on a specific date, thereby mimicking human-like memory with context and history. The implications for the AI/ML community are profound, as the new architecture not only enhances memory retention but also addresses critical scalability issues associated with query variability and latency. By separating vector and graph workloads into distinct stores optimized for their specific functions, CORE improved response times significantly—from as much as 9 seconds down to a target of 1-2 seconds. The project's insights highlight that effective memory systems must account for the evolving nature of information, thereby paving the way for more sophisticated AI-driven applications that can accurately reflect changes in data over time.