Meta's Infrastructure Evolution and the Advent of AI (engineering.fb.com)

Meta outlines how the rise of AI has forced a wholesale rethink of two decades of infrastructure design: from early LAMP-era scaling of web services and distributed caches to a global fabric of data centers, hundreds of edge POPs, and bespoke fleet-management and reliability systems (Twine, Tectonic, ZippyDB, Shard Manager, Delos, Service Router, Kraken, Taiji, Maelstrom). The company emphasizes its continuing open-source and open-standards stance as it moves down the stack into custom silicon and hardware systems. The core message: AI workloads change every assumption about scale, latency and failure modes — and require co-design across hardware, networks, data centers and software. Technically, Meta documents the transition from CPU-oriented, failure-tolerant web fleets to high-performance GPU clusters built for large-scale model training. Early AI clusters were ~4k GPUs and training jobs grew from ~128 GPUs to thousands as LLMs emerged; Meta scaled synchronous jobs to 2k–4k GPUs and then built two 24k-H100 clusters (one using InfiniBand, one RoCE) sized to consume a data center’s low‑tens-of-megawatts. Challenges included single-GPU faults stalling synchronous training, network jitter and memory errors; Meta reports a ~50x drop in interruption rates through engineering and partner work. They also developed model-level optimizations (e.g., HSTU) that can accelerate generative recommenders 10–1,000x. The account underscores that continuing progress in LLMs and personalized AI will demand larger, lower-latency fabrics, tighter fault tolerance, and integrated hardware-software innovation.