Show HN: I built an LLM that never forgets – persistent user memory with RAG (github.com)

A developer released an open-source FastAPI “proxy” that gives any OpenAI-compatible LLM persistent, autonomous user memory by combining RAG-style retrieval with PostgreSQL + pgvector. On each user message the system retrieves relevant memories by vector similarity, then makes a single LLM call that both generates the conversational response and extracts structured facts (JSON) about the user—automatically categorizing them (preference, personal, skill, goal, opinion, experience, other) and scoring importance (0.5–2.0). Only extracted facts are saved (not full conversations), memories persist across sessions, and the proxy returns {response, facts} for easy downstream use. Technically, it uses google/embeddinggemma-300m embeddings (default vector dim issues noted), IVFFlat indexes for fast similarity search, and a simple user_memories schema (content, embedding VECTOR(768), memory_type, importance, created_at). It’s optimized for cost and latency with one LLM API call per message (example vLLM or any compatible model), JSON validation, and hooks to tune max_tokens or swap in stronger LLMs (e.g., Llama 3). Practical caveats: you must run an LLM API endpoint, ensure embedding dimensions match the DB, and may need prompt/tokens tuning to avoid parse errors. MIT-licensed and Docker-friendly, it’s a pragmatic template for building long-term, context-aware conversational agents.