Tri Dao: The End of Nvidia's Dominance, Why Inference Costs Fell (youtu.be)

In a wide-ranging conversation on the Unsupervised Learning podcast, Tri Dao maps the current AI hardware landscape—Nvidia’s dominant position, emerging competitors, and why chip design is getting harder and more expensive. He stresses that progress now depends less on raw transistor scaling and more on smart hardware–software co‑design: compiler optimizations, quantization, kernel fusion, and runtime scheduling that squeeze orders of magnitude more inference throughput out of existing silicon. Tri covers innovations in AI-specific hardware, the rising importance of hardware abstractions, and how reinforcement-learning and fleet‑level optimization can coordinate batching, model placement, and latency/throughput tradeoffs across datacenter fleets. The discussion highlights practical implications for researchers and engineers: specialization (inference-optimized accelerators and kernels) will be critical for cost-effective deployment, while open-source tooling and better abstractions lower the barrier to exploit hardware advances. Looking ahead, Tri flags agentic workloads and real‑time video generation as drivers of new architecture patterns (multi‑resolution processing, robotics integration) and calls for close collaboration between academia and industry to iterate on architectures, compilers, and runtime systems that meet these real‑time, heterogeneous demands.