Dynamic Tool Allocation for AI Agents (The Rats Pattern) (rewire.it)

AI researchers introduced the RATS (Retrieval-Augmented Tool Selection) pattern to solve "tool overload" in agent systems: giving an LLM a long static list of tools bloats prompts (500–1,500 tokens per tool, leading to 25k–75k tokens at scale), increases latency and cost, and causes selection errors, API hallucinations, and brittle behavior. RATS applies a "select, then execute" architecture: a lightweight router agent first retrieves a small, highly relevant subset of tools for a query, then a stronger specialist agent executes the task using only that curated set. This reduces token usage, improves tool-selection precision, lowers API costs/latency, and scales to large tool catalogs; it also pairs well with State-Based Gating to pre-filter tools by workflow state. The post gives a production-aware Google ADK implementation: an orchestrator (gemini-1.5-flash) that must call a ToolRetrieverTool, which returns top-k tools above a confidence threshold, and an on-the-fly action agent (gemini-1.5-pro) built with only the retrieved FunctionTools (which return structured dicts and use try/except). Crucial details include session isolation to avoid context pollution, semantic/top-k retrieval (demo uses simple scoring but should use a vector DB like Vertex AI or Pinecone with hybrid semantic+BM25 search), and production practices like richer ranking signals, access control, and robust error handling.