Inside Agentic AI Workloads (gimletlabs.ai)

A recent collaboration between Gimlet Labs, the University of Illinois at Urbana-Champaign (UIUC), and Intel has revealed significant insights into the efficiency of ReAct-style agent workloads. Unlike traditional AI systems that optimize individual model requests, the study emphasizes the importance of understanding the entire execution trajectory of AI agents, which engage in iterative interactions, maintain growing contexts, and call upon various tools throughout their tasks. This research, presented at the 2026 IEEE International Symposium on Workload Characterization, indicates that serving systems must adjust their strategies to manage resources effectively across these complete trajectories, considering both cached states and scheduling. Key findings show that agent workloads require more than just handling long prompts; they consist of multiple model invocations where context builds over time. The research highlights how KV-cache retention can drastically enhance performance by allowing agents to reuse existing context, thus reducing computational costs associated with frequently invoked model calls. Additionally, tool usage evolves throughout the agent's execution, reflecting shifts in task intent that can inform resource allocation more effectively. Consequently, the authors argue for a trajectory-level control layer within AI infrastructure that tracks cumulative resource usage and optimizes for the entire task flow rather than individual requests. This approach aims to enhance the efficiency of AI agent serving, paving the way for more sophisticated applications in software development and data analysis.