Cloud-native computing is poised to explode, thanks to AI inference work (www.zdnet.com)

CNCF leaders told KubeCon that AI is shifting from a training-centric model to “enterprise inference,” and cloud-native infrastructure is primed to absorb that shift. Jonathan Bryce and CTO Chris Aniszczyk argued that instead of every company chasing billion-dollar LLM training runs (GPT‑5 training was cited as possibly costing up to $1B), organizations will deploy hundreds of smaller, fine‑tuned open models for specific tasks and serve them via cloud-native inference engines. A new wave of platforms — KServe, NVIDIA NIM, Parasail.io, AIBrix, llm-d and others — use containers and Kubernetes to deploy, scale and manage inference in production, turning models into responsive services that feed intelligence into apps and systems. Technically, the move matters because inference workloads favor cost, latency and privacy tradeoffs: smaller models are cheaper to run and fine‑tune, often faster or more accurate for narrow domains, and can be self-hosted to meet security needs. Kubernetes is evolving (dynamic resource allocation, GPU/TPU abstraction) to handle heterogeneous accelerator scheduling, and “neoclouds” are emerging to offer GPUaaS and bare‑metal stacks optimized for inference. Usage metrics (Google’s internal inference jobs rose to ~1.33 quadrillion tokens/month) and CNCF forecasts suggest enterprises will pour hundreds of billions into cloud‑native AI inference soon, spawning Inference‑as‑a‑Service offerings and making platform engineers central to enterprise AI adoption.