Why I'm Bearish on Model Routers (twitter.com)

In a recent discussion, a Delphi engineer expressed skepticism about the effectiveness of model routers, tools designed to optimize the use of coding agents and reduce token costs. While many in the AI/ML community view these routers as a way to delegate simpler tasks to cheaper models and cut expenses, the engineer pointed out a significant gap: engineers prioritize speed and efficiency over cost-saving measures that could slow down their workflow. The author argues that engineers often prefer familiar, high-performance models, disregarding the potential cost advantages of using a diverse range of routers. Furthermore, in scenarios involving "software factories" where engineers are removed from the coding process, the engineer believes that the control over model selection becomes critical. Since these factories can gather detailed performance data, organizations might prefer designing their own systems that allow for tailored model selection rather than relying on a third-party router that lacks insight into the specific needs of their codebases. This skepticism about model routers highlights a fundamental mismatch between management's cost-cutting goals and engineers' priorities, paving the way for deeper discussions within the AI/ML community about balancing efficiency and cost in software development workflows.