From LLM Inference to Agentic Workloads: Characterization and Implications (arxiv.org)

A new study has introduced AgentSysBench, a benchmark suite designed to characterize the system behavior of "agentic applications" that utilize large language models (LLMs) in more complex, long-running workflows. Unlike traditional LLM tasks that focus on isolated inference, these agentic workloads involve coordinating various tools and maintaining persistent state, fundamentally altering performance dynamics. The research identified six key properties differentiating these workloads from conventional serving, such as significant stateful execution, diverse resource affinities impacting latency, variable bottlenecks, and opportunities for caching redundancies in production traces. This work is particularly significant for the AI/ML community as it addresses the inadequate understanding of the system behaviors associated with agentic workloads, which are increasingly common in real-world applications. The findings suggest actionable improvements, including task-aware serving techniques that could reduce latency by up to 40% and state offloading that minimizes memory usage. By emphasizing the varied performance demands of agentic applications, this research paves the way for more efficient AI serving systems that can better handle the complexities of modern LLM deployments.