Orthrus-Qwen3: up to 7.8×tokens/forward on Qwen3, identical output distribution (github.com)

The newly announced Orthrus-Qwen3 framework combines the precise generation of autoregressive Large Language Models (LLMs) with the rapid parallel token generation of diffusion models. Utilizing a Qwen3 backbone, this innovative approach claims to provide a significant inference acceleration, achieving up to a 7.8× speedup in generation tasks while maintaining strictly lossless output that matches the original model's predictive distribution. The Orthrus models demonstrate enhanced efficiency by allowing both autoregressive and diffusion components to utilize the same high-fidelity Key-Value (KV) cache, promoting quicker processing with minimal additional memory overhead. Significantly, Orthrus outperforms existing speculative decoding methods like EAGLE-3 and DFlash, which often suffer from reduced accuracy and memory inefficiencies. By fine-tuning only 16% of the total model parameters while keeping the base LLM fixed, Orthrus enables faster and more reliable generation, especially on complex tasks. This dual-architecture framework sets a new benchmark for speed and performance in natural language processing, marking a crucial advancement for the AI/ML community in improving both the scalability and fidelity of model outputs.