WeDLM Reconciling Diff Lang Models with Std Causal Attention for Fast Inference (github.com)

Tencent's WeChat AI has introduced WeDLM, a new approach to diffusion language models that tackles the limitations of existing models by employing standard causal attention. This innovation allows for parallel mask recovery while maintaining key-value (KV) cache compatibility, which significantly enhances inference speed without sacrificing accuracy. Notably, WeDLM can directly initialize from pre-trained autoregressive (AR) models, such as Qwen2.5 and Qwen3, making it easier for developers to leverage its capabilities. The significance of WeDLM lies in its ability to improve efficiency for tasks with structured outputs, achieving speedups of 3-6 times for math reasoning and 2-3 times for code generation compared to production-grade engines like vLLM. The architecture utilizes topological reordering and streaming parallel decoding to optimize performance. This new method addresses the prevalent issue of translation between parallel prediction and actual processing speeds in optimization frameworks, marking a considerable advancement in the AI/ML community's quest for faster and more efficient model deployments. The potential trade-offs between speed and quality are also acknowledged, allowing users to fine-tune performance based on their specific requirements.