General plug-and-play inference lib for RLMs (github.com)

The Recursive Language Models (RLMs) library has been announced as a novel approach to enhance language model inferences by effectively managing extensive context lengths. By allowing language models to decompose and recursively process their inputs, RLMs introduce a new methodology that replaces traditional LLM prompts with a programmable completion call. This innovative system provides a flexible inference engine compatible with both API-based and local language models, supporting various sandbox environments for secure execution. This development is significant for the AI/ML community as it opens up new possibilities for handling complex language tasks while ensuring safety and modularity in execution. The library offers two types of REPL environments, including isolated cloud-based sandboxes, which mitigate risks associated with executing potentially unsafe code. The initial experiments and concepts behind RLMs were formalized in a blog and documented in a preprint, paving the way for ongoing collaboration and enhancements within the open-source community. With its compatibility with major clients and improved logging and visualization tools, the RLMs library represents a promising step forward in the evolution of language models and their applications.