vLLM v0.28.0 (github.com)

The recent release of vLLM v0.28.0 marks a significant update with 584 commits from 270 contributors, introducing major performance enhancements and new features that will impact the AI/ML community. A key highlight is the Kimi-K3 optimization effort, which includes support for Decode Context Parallel (DCP), increased kernel speeds, and adaptive speculative decoding techniques. These improvements collectively enhance model performance, with a reported 1.5-3x speedup for certain operations and a ~60% boost in token processing efficiency. The update also expands hardware compatibility, allowing Kimi-K3 to run on AMD ROCm platforms, fostering wider accessibility for developers. Additionally, vLLM v0.28.0 improves model management through features like Model Runner V2 disaggregation and enhanced tiered KV cache offloading capabilities, which streamline resource use for large-scale deployments. New model support includes advanced versions of Muse Glimmer and Qwen, alongside significantly improved speculative decoding processes. With these enhancements, the release not only optimizes existing workflows but also sets the stage for advanced use cases in multimodal AI, reinforcing vLLM's positioning as a versatile backbone for AI model serving and optimization.