We decreased our LLM costs with Opus (www.mendral.com)

A recent update revealed that the Opus 4.6 architecture has significantly reduced operational costs compared to the previous Sonnet 4.0 setup, achieving greater efficiency in processing Continuous Integration (CI) logs. The key advancement lies in a new triager agent, Haiku, which filters out irrelevant issues, allowing only 20% of failures to escalate to Opus for in-depth investigation. This dual-layer model optimizes resource allocation by ensuring that straightforward duplicates don’t burden the more expensive model, thus offering a cost-effective solution while maintaining high accuracy in diagnosis. Notably, the Haiku agent employs a combination of error message attachment and semantic search to quickly identify whether an issue has been previously reported. It utilizes a SQL interface to dynamically retrieve relevant log data, rather than pushing entire logs into the model, which streamlines the analysis process. This architecture allows Opus to hypothesize and guide sub-agents in a structured manner, ensuring clean context management and directed exploration of the CI logs. Overall, this fluent integration of cost-efficient components not only enhances performance in CI scenarios but also has implications for other high-volume data fields such as security and financial logging.