Training Large Language Models to Reason in a Continuous Latent Space [pdf] (arxiv.org)

Researchers have introduced a novel approach for training large language models (LLMs) called Coconut (Chain of Continuous Thought), which shifts from traditional reasoning in language spaces to a continuous latent space. This paradigm aims to enhance LLMs' reasoning capabilities by moving away from the limitations of token-based expressions, allowing models to represent complex reasoning states directly through their last hidden layer. Instead of generating words as outputs, Coconut reuses the latent reasoning state as input for subsequent processes, enabling more flexible reasoning strategies that incorporate multiple potential pathways. This advancement is significant for the AI/ML community as it addresses the challenges that LLMs face in tasks requiring intricate logical reasoning and planning. By implementing a breadth-first search (BFS) strategy, Coconut shows improved performance over traditional chain-of-thought methods, especially in tasks where exhaustive exploration of alternatives is crucial. The switch to continuous thought representations not only enhances the efficiency and accuracy of LLMs but also opens new avenues for developing smarter, more adaptable AI systems capable of more complex tasks than previously possible.