Running Two 27B Models on a Single RTX 5090 (blog.kvit.app)

A new approach allows running two 27-billion-parameter (27B) models—Qwen3.8 and Cloudflare’s Clef—simultaneously on a single RTX 5090 graphics card. This was achieved by extracting a LoRA adapter from Clef and modifying the vLLM framework to serve both models concurrently, achieving rapid responses even for large prompts. For instance, Clef can now process requests in as little as 150-310 milliseconds for familiar prompts while maintaining an efficient chat model through shared resources. This development is significant for the AI/ML community as it highlights advanced resource efficiency, enabling substantial concurrent model functionality without the need for additional hardware. The results show a slight trade-off in chat capabilities—reducing the number of simultaneous conversations from eight to six and the context size—yet maintaining impressive processing speeds. This innovative setup will enhance applications in interactive AI scenarios, such as coding assistants and decision-making systems, demonstrating the potential for integrating multiple models seamlessly while optimizing performance.