Multi-Token Prediction Done Right (arxiv.org)

A recent breakthrough in Multi-Token Prediction (MTP) training has been announced, crucial for enhancing language models' performance and speeding up inference processes. Researchers identified significant flaws in traditional MTP methods, where residual gradients could negatively impact both next-token and multi-token learning. They proposed a novel "KL gradient-recovery trick" that enables the extraction of KL divergence gradients without needing extensive vocabulary matrices. This innovation leads to "Soft Multi-Token Prediction" (Soft-MTP), which replaces conventional MTP with a straightforward one-line code modification to optimize the learning process. Soft-MTP significantly enhances model performance, achieving lower perplexity in next-token predictions and faster speculative decoding across a range of model sizes from 167M to 1.2B parameters. It not only maintains the advantages of DeepSeek and Meta-style MTP but also leads to improved zero-shot accuracy and superior reasoning capabilities in math and coding tasks upon fine-tuning. Essentially, Soft-MTP offers a cost-free optimization technique for MTP, accomplishing results comparable to direct KL minimization while simplifying the training pipeline. This advancement promises to streamline language model training in the AI/ML community, making it a noteworthy development in the field.