Memory chips could be the next bottleneck for AI (2024) (economist.com)

The semiconductor market is showing fresh fault lines: while equipment supplier ASML saw orders plunge to half analyst expectations, TSMC reported record profits, and memory-maker SK Hynix has cemented a dominant position in the memory-chip market. That concentration in DRAM and high-bandwidth memory (HBM) suppliers, paired with volatile upstream demand, has raised alarms that memory — not logic chips or fabs — could become the next systemic bottleneck for AI workloads. For AI/ML teams, this matters because modern training and inference are increasingly constrained by memory capacity and bandwidth as model sizes grow. Tight supply or price swings for DRAM/HBM could slow model scaling, increase costs for hyperscalers and startups, and accelerate engineering workarounds: more aggressive quantization and sparsity, memory-efficient architectures, distributed training strategies, on-chip memory innovations, or investment in vertical supply security. In short, the market dynamics around SK Hynix and uneven semiconductor demand suggest the industry may need to rebalance hardware, software and procurement strategies to avoid memory limits becoming the rate‑limiting step for future AI progress.