Building the Analytics Agent on Metabase: A Progress Report (medium.com)

A developer reported progress building an “Analytics Agent” that augments Metabase to translate natural-language questions into accurate, production-ready SQL. Starting with a RAG pipeline (metadata extracted from Metabase, docs indexed in a vector store like Pinecone, and an LLM generating SQL), they ran into real-world gaps: generated queries missed business nuance, didn’t probe data, and struggled with large schemas. Choosing Metabase was deliberate — its metadata API, self-hosting model, and familiar UI let the agent augment existing BI workflows without moving data to a SaaS. To fix this they moved to an agentic design where the assistant calls explicit tools (Database Overview, Relationship Explorer, Table Inspector, Query Tester), runs read-only exploratory queries, and reasons iteratively. They added short- and long-term memory to cache findings across sessions and a human-in-the-loop clarifying dialog to resolve ambiguous business logic (e.g., what “active user” means). Results show meaningful gains: ~50% higher first-try SQL accuracy on one benchmark, response time trimmed from ~65s to ~58s, and token use down ~75%. The takeaway for AI/ML teams: LLMs plus schema context aren’t enough — tool use, probing the data, memory, and user clarification are essential to scale reliable text-to-SQL systems for complex, real-world databases.