Unlocking Lossless Speedups in LLMs via Discrete Diffusion (5000 Tk/S) (s-sahoo.com)

Researchers have unveiled a breakthrough in large language models (LLMs) with the introduction of diffusion-augmented LLMs, specifically the new model called Uno. This innovative architecture combines two sets of weights – standard autoregressive (AR) weights for traditional model training and lightweight diffusion weights designed to generate multiple tokens simultaneously. This dual-weight system allows Uno to deliver lossless speedups in inference, achieving throughput rates that surpass leading speculative decoding technologies, such as DFlash and Eagle3, while maintaining low memory usage and minimal additional parameters. The significance of this development lies in Uno's ability to improve efficiency without sacrificing performance, achieving up to three times the speed of standard AR models and outpacing competitors like the 26B model DiffusionGemma across various benchmarks, including coding and long-context reasoning tasks. The introduction of \(\Psi\)-Spec samplers enables the generation of tokens in parallel, marking a step forward in optimizing LLMs for real-world applications. This progress could reshape the landscape of AI/ML by enabling faster, more efficient models that can handle complex tasks with greater ease and accuracy.