Recursive Language Models (arxiv.org)

A recent study has introduced Recursive Language Models (RLMs), a novel inference strategy allowing large language models (LLMs) to process significantly longer prompts than previously possible. RLMs enable LLMs to treat long prompts as part of their external environment, allowing them to programmatically analyze, decompose, and recursively call upon different segments of a prompt. This approach enables RLMs to effectively handle inputs that are up to 100 times longer than traditional model context windows. The significance of this innovation lies in its ability to dramatically enhance the quality of responses for various long-context tasks, surpassing the performance of baseline LLMs and existing long-context workarounds. Not only do RLMs improve output quality, but they also maintain similar or even reduced costs per query. This advancement marks a pivotal development for the AI/ML community, as it opens up new possibilities for applications requiring complex, extended contextual understanding and lays the groundwork for more sophisticated and scalable LLM interactions in real-world scenarios.