Cost-Efficient Pareto Recommendation for LLM Agentic Workflows (arxiv.org)

A new framework called Cost-Coupled Gittins (CC-Gittins) has been introduced to enhance decision-making in large language model (LLM) agentic workflows by balancing task quality, deployment costs, and latency. This multi-objective approach allows users to find optimal model assignments tailored to their specific needs while effectively minimizing the search costs traditionally associated with evaluating multiple model candidates. By leveraging the relationship between evaluation and deployment costs, CC-Gittins alternates its focus between maximizing quality and minimizing costs, resulting in more efficient recommendations. The significance of CC-Gittins lies in its ability to recover a substantial percentage of the true Pareto frontier—representing the best possible trade-offs between quality and cost—while bearing only about 10% of the exhaustive evaluation cost. This efficiency is particularly relevant as workflows become increasingly complex, involving multiple roles and numerous candidate assignments, which can reach up to 1,681 options. Importantly, CC-Gittins also accommodates latency as a third objective, making it a versatile tool for optimizing LLM agentic workflows in real-world applications.