Reactive Transformer Research Paper (huggingface.co)

Reactive Transformer (RxT) is a new event-driven architecture that makes Transformers stateful and real-time by adding a fixed-size Short-Term Memory (STM) and decoupling generation from memory updates. Instead of reprocessing an ever-growing conversation history, RxT treats each turn as an event: a generator-decoder produces a response using the current query plus the previous STM state, and then—asynchronously—a memory-encoder and a Memory Attention network compress the full interaction into the STM. That design yields constant per-message compute and memory use and removes prompt-phase latency, turning the usual quadratic conversation cost into linear scaling (paper cites a reduction from O(N^2 · T) to O(N · T) with N interactions). For the AI/ML community this is significant because it provides a practical route to long, low-latency, stateful dialogues without blowing up compute or response time—important for real-time agents, multi-turn assistants, and edge deployments. The paper reports proof-of-concept experiments on synthetic data where RxT outperforms a same-size stateless decoder model and achieves constant-time inference. Caveats: results are small-scale and synthetic, so broader benchmarks, robustness tests, and integrations with retrieval or long-term memory will determine how well RxT generalizes in production systems.