FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion LMs (machinelearning.apple.com)

Researchers introduced FS-DFM (Few-Step Discrete Flow-Matching), a new diffusion-style language model that delivers fast, high-quality long-text generation by explicitly training for very small sampling budgets. Unlike autoregressive models that generate tokens serially, diffusion language models can parallelize across token positions but have historically required hundreds–thousands of iterations to match ARM quality. FS-DFM makes the number of sampling steps an explicit parameter, trains the model to be consistent across different step budgets, and uses distilled teacher guidance from long-run trajectories so a single large update lands where many small updates would. On language modeling benchmarks, an 8-step FS-DFM matches the perplexity of a 1,024-step discrete-flow baseline for generating 1,024-token sequences, yielding up to 128× faster sampling and corresponding latency and throughput gains with a similar-size model. Technically, FS-DFM combines a discrete flow-matching objective with a stable update rule that moves probability mass in the correct direction without overshooting, plus strong trajectory distillation to stabilize few-step sampling. The result is few-step sampling that is stable, accurate, and controllable—bringing diffusion LMs into practical territory for long-form generation and real-time applications. This approach suggests diffusion methods can rival autoregressive throughput while preserving quality, opening a path to scalable, parallel token generation for large-context tasks.