Frontier: A Discrete-Event Simulator for Modern LLM Serving (github.com)

Frontier, a new discrete-event simulator designed for modern large language model (LLM) serving, has been announced with its initial version set for release in June 2026. This simulator is tailored for complex serving systems that require sophisticated optimizations, including support for sparse model architectures and stateful workloads. Currently centered on the vLLM engine, Frontier aims to facilitate research and engineering efforts by providing insights into serving system designs without the expense and time investment of extensive GPU deployments. Notably, it supports co-located serving in a monolithic cluster environment, with plans for disaggregated architecture capabilities in the future. The significance of Frontier lies in its ability to accurately model advanced production techniques such as CUDA Graph and hierarchical caching, allowing users to explore designs under specific service level agreements (SLAs) and making deployment decisions more informed. By simulating runtime behavior rather than relying solely on average metrics, it enables a more nuanced understanding of performance impacts. Frontier requires minimal hardware resources, operating entirely on CPU machines except for profiling, thus streamlining what-if analyses that would typically demand heavy GPU computation. This tool promises to push the boundaries of LLM serving efficiency and performance in the AI/ML community.