How ZigZag made our workflow worse before we fixed it (zagforge.com)

ZigZag, a CLI tool designed to optimize workflows with large language models (LLMs), initially failed to deliver on its promise of reduced token consumption while navigating codebases. The team initially believed that a condensed Abstract Syntax Tree (AST) report would streamline access to code by replacing extensive file browsing with a structured overview. However, upon measuring its performance against native tools like ripgrep, they found that ZigZag's report was larger than the original code, ultimately increasing costs rather than decreasing them. This inefficiency stemmed from the additional metadata and structural indexes introducing overhead. To address these issues, the team restructured ZigZag's approach, focusing on distinguishing between code structure and code bodies. By implementing an `llm-signatures` mode that retained key information about declarations while omitting the associated code snippets, they achieved a more efficient token usage. The redesigned tool could now map a codebase accurately for around 60,000 tokens compared to 67,000 tokens required by previous methods. This pivot has positioned ZigZag as an effective resource for onboarding newcomers to unfamiliar codebases, emphasizing the importance of continuously measuring tools' performance against their intended outcomes to avoid unexpected failures.