First Principles of Model Routing (try.works)

A new set of principles for model routing has been proposed by the developer of a model router called role-model. These guidelines emphasize the importance of keeping models distinct and maintaining a manageable pool size for effective routing. The first principle suggests that routing between similar generalist models, like GPT and Opus, can complicate decision-making, as they perform comparably across tasks. Instead, leveraging a combination of a high-performance model and one that excels in specific constraints—speed, quality, or cost—enhances routing efficiency, facilitating clearer decisions when directing requests. The second principle advocates for a limited model pool, with a recommendation to stick to two models initially. This avoids the confusion that can arise from adding numerous models with overlapping capabilities. Finally, it stresses the importance of using real-world benchmarks that accurately reflect specific workloads, rather than relying solely on metadata or generic performance metrics, which may not provide a reliable basis for routing decisions. These principles are designed to optimize model routing, ultimately enhancing workflow efficiency within the AI/ML community.