Most AI Work Can Wait (tomtunguz.com)

A recent discussion in the AI community emphasizes the importance of designing agent systems around task routing rather than model selection. Rather than prioritizing which AI model to use first, it's crucial to establish an efficient routing mechanism that determines how tasks are handled. A well-implemented router allows teams to manage up to 70-80% of AI traffic using local models or asynchronous processing, significantly cutting costs—potentially by over 90%. This approach enables organizations like Coinbase to scale their AI capabilities while keeping expenses stable, highlighting the role of routing and defaults over friction in achieving efficiency. The routing strategy involves three layers: a skill classifier that identifies the task type, a router that assigns the task to the appropriate model tier based on features like complexity and historical success, and a model selector that finds the cheapest model fulfilling the task requirements. By prioritizing the design of the routing system, AI engineers can maximize performance while minimizing costs, especially since many tasks, such as drafting replies or summarizing documents, do not require immediate responses. This shift in focus towards a well-structured routing solution may redefine how AI systems are built, encouraging teams to consider the design of their systems before making model choices.