I Rented a 96 GB GPU and Took Uncensored Qwen3.8 From 44 to 125 tok/s (aseemshrey.com)

A researcher recently rented an RTX PRO 6000 GPU with 96 GB of VRAM to optimize the performance of the Qwen3.8-27B-Ucensored model, achieving a remarkable increase in token generation speed from 44 tok/s to 125 tok/s. The experiment utilized self-hosted orcarouter/Qwen3.8 through vLLM, which allowed the researcher to streamline the model's execution without interruptions or the typical moral restrictions often encountered with other AI systems. This autonomy is particularly significant, enabling more efficient and straightforward interactions with the model during complex tasks like the Ox Alpha investigation. The technical implications of this experiment are noteworthy for the AI/ML community. The use of the 96 GB card facilitated advanced features like FP8 precision and a considerable context length of 262K tokens, maximizing the model’s capabilities without running into memory limitations. The choice of vLLM’s MTP3 and DFlash2 profiles further enhanced output speed, demonstrating a 72% improvement and a staggering 53% increase, respectively, compared to earlier benchmarks. This research not only highlights the potential efficiencies in large-scale AI deployments but also underscores the ongoing need for models that allow for more control and customization in their execution environments.