Consistency diffusion language models: Up to 14x faster, no quality loss (www.together.ai)

Seoul National University and UC Berkeley researchers have announced the development of Consistency Diffusion Language Models (CDLM), a significant advancement that enhances the efficiency of inference in diffusion language models (DLMs) by up to 14.5 times without sacrificing quality. This improvement is achieved through a combination of consistency-based multi-token finalization and block-wise key-value (KV) caching, addressing two major inefficiencies inherent in traditional DLMs. Specifically, these models often require high computation due to their bidirectional attention mechanism and extensive refinement steps, which can slow down performance. CDLM optimizes the inference process by focusing on fewer refinement steps while still maintaining high output quality. It utilizes a novel post-training approach that enforces trajectory-consistent behavior, enabling models to finalize multiple tokens simultaneously during inference and effectively cache prior computations. Experimental results show CDLM achieving substantial reductions in processing time, indicating its potential to enhance throughput in tasks such as math and coding. This advancement positions diffusion models as increasingly competitive alternatives to autoregressive models, highlighting a pivotal shift towards more efficient deep learning architectures in the AI/ML community.