Docker model runner integrates vllm (www.docker.com)

Docker Model Runner now integrates the high-throughput vLLM inference engine and safetensors model support, letting developers run production-grade LLMs with the same Docker workflow they use for smaller experiments (like llama.cpp). vLLM brings PagedAttention for lower memory overhead, native batching and streaming, and compatibility with popular open-weight models (GPT-OSS, Qwen3, Mistral, Llama 3) in safetensors format. Docker Model Runner automatically routes jobs to the appropriate backend based on model format (safetensors → vLLM, GGUF → llama.cpp), so CLI/API usage stays identical (e.g., docker model install-runner --backend vllm --gpu cuda and unchanged HTTP endpoints). This unification matters because it removes the tradeoff between ease-of-use and throughput: prototype locally on llama.cpp and scale to Nvidia GPUs with vLLM without changing CI/CD or container deployments. The initial release targets x86_64 systems with Nvidia GPUs and supports publishing models as OCI images. Known caveats include vLLM’s slower startup/time-to-first-token versus llama.cpp, which Docker plans to optimize, plus upcoming work for WSL2/Docker Desktop and Nvidia DGX compatibility. For teams building scalable inference pipelines, this offers a simple path from laptop experimentation to high-throughput production serving.