ByteDance's new Diffusion LLM beats other dLLMs (seed.bytedance.com)

ByteDance’s Seed team has published Seed Diffusion Preview, an experimental “discrete diffusion” language model targeting structured code generation. The release demonstrates that diffusion-based LLMs can match autoregressive models’ code quality while massively improving inference throughput: the model achieves 2,146 tokens/sec—about a 5.4× speedup over similarly sized autoregressive baselines—establishing a new speed-quality Pareto frontier for code models. The work is billed as a proof-of-concept that discrete diffusion can serve as a foundational framework for next-generation LLMs. Technically, the team combines several innovations to make diffusion viable for language tasks: two-stage diffusion training, constrained order learning to manage token generation order, and on-policy learning to enable efficient parallel decoding. Experiments across eight open code benchmarks (including LiveCodeBench’s 1,055 problems from v1–v6) show comparable core-benchmark performance despite the dramatic speed gains. The paper also notes caveats about cross-model speed comparisons—baselines used different datasets, hardware (e.g., H100s), and evaluation protocols—so absolute ranking may vary. Still, Seed Diffusion’s results are significant for the AI/ML community because they validate a scalable, parallelizable decoding path for LLMs that could reshape trade-offs between latency and quality in code and other structured generation tasks.