Recycler: Cost-Free Latent Reasoning via Hidden State Recycling [pdf] (lf3-static.bytednsdoc.com)

ByteDance Seed has introduced Recycler, a novel architecture that enhances autoregressive language models by incorporating a mechanism that recycles hidden states during text generation. Unlike standard models that discard previous hidden states after each token is generated, Recycler retains and integrates these continuous hidden states along with token embeddings, significantly improving the model's capability in reasoning and mathematics tasks with just 2% additional parameters. This approach minimizes losses and maintains equivalent decoding costs compared to traditional transformers, thus achieving higher efficiency without the need for excessive parameter scaling. The significance of Recycler lies in its potential to optimize computational resources while enhancing model performance. By leveraging hidden states during autoregressive decoding, which typically represents a bottleneck in information propagation, Recycler creates a continuous feedback pathway that effectively deepens the contextual understanding of each generated token. This technical advancement paves the way for more efficient training and inference processes, making Recycler a promising development for the AI and machine learning community as it aims for greater model efficiency and accuracy in managing complex tasks.