Parallel Language Models via Variational Decoding (arxiv.org)

A recent study has introduced Parallel Language Models (PLMs), which address the high inference latency commonly associated with autoregressive language models that generate one token at a time. By leveraging latent auxiliary variables to manage dependencies, PLMs enable the generation of multiple, mutually consistent tokens in a single forward pass. This innovative approach is trained using a variational bound that combines autoregressive and parallel decoding losses, resulting in impressive performance metrics. Pretrained PLMs from models ranging from 80M to 3B parameters achieved results comparable to autoregressive models while significantly speeding up inference times by 2.3 to 5.7 times. This development is significant for the AI/ML community as it opens new avenues for more efficient language model training and deployment, facilitating faster and more scalable applications. The ability to maintain autoregressive quality during parallel generation and the provision of an open-source codebase will empower researchers and developers to utilize these models effectively. As PLMs are positioned on a favorable Pareto frontier compared to existing baselines, this may lead to broader adoption in various AI-driven tasks, from coding to instructional dialogue systems.