Using static analysis to reduce token usage of AI coding agents (faraazahmad.github.io)

A new approach to improving AI coding agents tackles their significant inefficiency in token usage caused by repetitive, text-based searches through codebases. Traditional AI agents lack persistent context and rely on keyword searches like grep and ls, which are brittle and often waste thousands of tokens scanning irrelevant files due to poor naming conventions or scattered code organization. This "context rot" not only inflates token consumption but also degrades model performance as context windows grow excessively large, limiting scalability on real-world projects with thousands of functions. To address this, the author proposes integrating classical computer science techniques with modern AI capabilities: summarizing functions using large language models (LLMs) to produce semantic summaries and generating vector embeddings of these summaries. These embeddings enable efficient semantic similarity searches—powered by PostgreSQL’s pgvector extension—that find relevant functions based on meaning rather than brittle keyword matches. Additionally, constructing detailed call graphs stored in graph databases like Neo4j helps trace complex function dependencies quickly without token-heavy exhaustive searches. Benchmarks show this hybrid approach cuts token usage by 30-54% on tasks such as identifying functions managing state or parsing templates, dramatically reducing computational costs and latency. This work highlights a crucial insight for the AI/ML community: while LLMs excel at language understanding, coupling them with targeted static analyses, semantic embeddings, and graph databases unlocks practical efficiency gains and scalability in coding assistance. By reducing redundant token consumption and leveraging symbolic program structures, this method paves the way for smarter, more cost-effective AI agents better suited for large, messy codebases encountered in real development environments.