Autoresearch doubled GLM-5.2 throughput. Production traffic broke it (fparisio.substack.com)

T-Systems has successfully enhanced the throughput of the GLM-5.2 model by employing an AI autoresearch agent on their NVIDIA B200 cluster, which initially doubled the throughput across 26 iterations. However, when subjected to real production traffic, the results revealed significant performance issues with long token time-to-first (TTFT) latencies. The team overcame these challenges through manual tuning and optimized configurations, achieving 18 times faster first tokens and a peak throughput of 442 tokens per second per GPU. These improvements were made possible by a strategic combination of architecture adjustments, including the utilization of NVFP4 quantization and sophisticated memory management techniques. The significance of this development lies not only in the substantial performance gains achieved but also in the innovative use of an autoresearch framework that leverages iterative learning to refine configurations autonomously. This approach allowed the system to intelligently exclude unproductive configurations early on, enhancing operational efficiency. As a result, T-Systems optimally balanced latency and throughput, demonstrating the potential for AI-driven techniques to revolutionize the deployment and scaling of large language models, paving the way for improved performance in real-world applications.