Doc-to-Atom: Learning to Compile and Compose Memory Atoms (arxiv.org)

Researchers have introduced Doc-to-Atom (Doc2Atom), a novel framework aimed at enhancing document understanding in Large Language Models (LLMs) by addressing the traditional limitations of long input sequences and high memory costs associated with attention mechanisms. Unlike previous methods that generate a single LoRA adapter for an entire document, Doc2Atom decomposes documents into semantically typed "knowledge atoms." Each atom is crafted into distinct micro-LoRA adapters and retrieval keys, enabling a query-specific composition of relevant atoms at inference. This innovative approach not only reduces the chances of query interference but also improves the scalability for reasoning over long documents. The significance of Doc2Atom for the AI/ML community lies in its ability to streamline multi-step reasoning while significantly lowering memory usage during inference. By utilizing a multi-objective distillation framework for end-to-end training, the system achieves superior performance across six diverse question-answering benchmarks compared to existing methods like Doc-to-LoRA. This advancement signals a noteworthy shift in how LLMs can manage and retrieve contextual information, heralding a more efficient and flexible approach for handling extensive information in natural language processing tasks.