Building the fastest LLMs: why we're starting with diffusion (celeris.ai)

Celeris has unveiled an ambitious strategy to create the world's fastest large language models (LLMs) by pioneering diffusion models tailored for extreme throughput from the outset. Traditionally, model design has prioritized training separate from real-time inference optimization, but Celeris is flipping this paradigm by integrating speed into both training and architecture from the ground up. The emphasis on diffusion models, known for their impressive parallel-generation capabilities in other domains, represents a significant shift in how LLMs are conceptualized, combining the coherence of sequential generation with the speed of parallel decoding. This approach aims to address notable limitations of existing autoregressive models, such as their struggle with reversal reasoning and commitment to each token during inference. Celeris's LLaDA model showcases the ability to attend to the entire sequence at once, outperforming GPT-4 on complex reasoning tasks. Moreover, their Diffusion-of-Thought framework allows models to self-correct during generation, enhancing accuracy and efficiency. By investing in neural architecture search, Celeris is building a research engine that continually evolves and optimizes model design for real-time applications. This paradigm shift could enable new software categories, including rapid voice interfaces and advanced reasoning agents, signaling a transformative leap in AI capabilities where latency no longer sacrifices model quality.