Running GLM-5.2 5x faster at 500tps with limitation (abhishek.it)

In June 2026, a benchmarking test revealed that the GLM-5.2 model can run approximately 5 times faster at around 480 tokens per second on Xiaomi MiMo's TileRT inference runtime, compared to vLLM's 96 tokens per second using the same hardware. While TileRT excels in speed, achieving this performance required overcoming multiple significant technical hurdles, including driver compatibility issues and the need to adapt the model's architecture for TileRT, which did not initially support GLM-5.2’s new IndexShare mechanism. This development is crucial for the AI/ML community as it highlights the ongoing challenge of optimizing large language models for faster inference while maintaining accuracy. The process revealed that while TileRT can handle short, latency-sensitive tasks effectively, it is constrained to a maximum context window of 2048 tokens, making it less suitable for applications requiring long-form outputs. In contrast, vLLM, despite its slower speed, can process the full context of GLM-5.2 without this limitation, positioning it as a better option for extensive text generation tasks. This benchmark not only demonstrates TileRT's significant speed advantage through innovative kernel design and adaptive remapping but also emphasizes the trade-offs developers must navigate between performance and model capabilities.