I accidentally turned LLM memory into program analysis (pwning.systems)

A recent exploration into enhancing large language models (LLMs) for vulnerability research led to the innovative development of Lemmalog, a Datalog engine designed to improve how LLMs maintain and update knowledge. The creator found that conventional memory systems for LLMs struggled during lengthy investigations, often resulting in erroneous suggestions based on outdated or incorrect assumptions. By framing the problem through the lens of program analysis, they sought to enable LLMs to automatically invalidate conclusions when foundational facts changed, akin to how dynamic program analysis allows for efficient updates without needing to reevaluate entire states. Lemmalog uniquely separates the LLM's role in understanding complex languages and code from the deterministic aspect of maintaining structured facts and rules. This allows the LLM to operate more like a front-end, processing input into structured format, while Lemmalog manages inference and fact derivation, tracking dependencies to maintain the integrity of conclusions. This innovative approach not only enhances the LLM's reliability during analysis by mitigating hallucinations but also provides a framework for provenance tracking and temporal facts, making it a promising advancement in the intersection of AI, ML, and program analysis.