ContextMemory – Markdown memory for your llama.cpp/vLLM server (github.com)

Kortexio has launched ContextMemory, a self-hosted memory gateway designed for servers running llama.cpp or vLLM. This innovative tool allows users to store session memory as markdown files, eliminating the need for vector databases or client-side rewrites. Clients can send just the new message in a conversation, while ContextMemory keeps track of session context seamlessly. This setup supports standard OpenAI-compatible requests and includes features like an admin UI, session wiki, and customizable LLM backends, empowering developers with flexibility over their AI infrastructures. The launch of ContextMemory is significant for the AI/ML community as it enhances how conversational AI systems manage memory, allowing for more efficient interactions without losing context across messages. By enabling a more straightforward approach to memory management through editable markdown files, ContextMemory facilitates both auditing and modification of stored information. Additionally, the innovative server-side processing supports integrations with various LLM engines and workflows, providing safeguards like validation and human-in-the-loop checks, all while maintaining ease of deployment via Docker. This enhances the robustness and control developers have over their AI applications.