Building a RAG on SQLite (blog.sqlite.ai)

SQLite now supports building a full retrieval-augmented generation (RAG) pipeline inside a single file thanks to two new extensions: SQLite-Vector (vector storage + similarity queries) and SQLite-AI (local model inference for embeddings and semantic tasks). The example project SQLite RAG shows how to ingest documents, generate embeddings via sqlite-ai, store vectors with sqlite-vector, and run hybrid searches that combine SQLite FTS5 full-text matches with semantic vector similarity. Queries are merged using Reciprocal Rank Fusion (RRF) so results that rank highly in either FTS or vector search are promoted. The tool is available as a compact Python module or CLI (sqlite-rag add /path/to/docs; sqlite-rag search "…"), keeping all search logic and vectors inside the DB. The demo targets edge and low-op environments: a GitHub Action builds the DB (182 docs, ~640 words each) producing chunk- and sentence-level embeddings in ~25 minutes on a standard runner, outputting one SQLite file. At runtime a lightweight server (4 vCPUs, ~100 MB memory) generates query embeddings with Gemma Embedding 300M Q8 and the Edge Function executes the hybrid search in ~370 ms. Significance: this brings vector search and local inference into an embedded, ops-free DB, improving privacy, simplicity and offline/edge viability. Next steps include text generation, multimodal support (images/audio), and further latency/quality optimizations.