Perplexitys First Research Paper – Point-to-Point Communication for LLM Systems (arxiv.org)

Perplexity’s paper introduces TransferEngine, a portable RDMA-style point-to-point communication layer aimed at LLM system patterns that need more flexible messaging than bulk collectives—examples include disaggregated inference (kv‑cache shuffles), Mixture‑of‑Experts (MoE) routing, and asynchronous RL fine‑tuning. The core idea is to expose a uniform, one‑sided WriteImm primitive plus an ImmCounter completion primitive that works across different NICs without depending on transport ordering guarantees, while transparently managing multiple NICs per GPU. That design avoids vendor lock‑in and makes low‑latency, fine‑grained transfers usable inside inference and training engines. Technically, TransferEngine demonstrates peak throughput of ~400 Gbps on both NVIDIA ConnectX‑7 and AWS EFA and integrates into three production workloads: dynamic kv‑cache transfer for disaggregated inference, RL weight updates that complete in ~1.3 s for trillion‑parameter models, and MoE dispatch/combine logic that beats DeepEP decode latency on ConnectX‑7 and achieves the first viable latencies on EFA. For the AI/ML community this matters because it complements collective primitives with portable point‑to‑point semantics, enabling scalable, low‑latency routing and asynchronous update patterns critical to modern LLM deployments without locking systems to a single NIC vendor.