What about OpenCL and CUDA C++ alternatives? (www.modular.com)

Longstanding efforts to create CUDA C++ alternatives—OpenCL, SYCL, oneAPI and others—sought to deliver portable GPU programming for AI, but they never became the dominant path for GenAI. The author, an OpenCL implementer at Apple, traces why: committee-driven standards moved too slowly, vendors kept new hardware features secret and pushed vendor-specific extensions, and there was no shared reference runtime, producing a patchwork of forks and weak conformance. Critically for AI, OpenCL never standardized support for modern accelerator primitives like Tensor Cores, which leads to large performance gaps (often 5x–10x slower than CUDA) and makes it impractical for costly GenAI training and inference. Meanwhile NVIDIA tightly co-designed CUDA libraries with TensorFlow and PyTorch, ensuring best-in-class performance and developer momentum—cementing CUDA’s dominance. For the AI/ML community this history matters because portability without performance is useless. The piece argues successful alternatives need a working reference implementation, strong stewardship, rapid evolution to match AI hardware and algorithmic change, robust tooling and developer experience, and a governance model that avoids fragmentation. It’s skeptical that slow, committee-based efforts can compete with vendor-driven ecosystems (and warns about vendor-controlled “open” projects like oneAPI). The author teases a follow-up on AI compiler stacks (TVM, OpenXLA, MLIR) as the next frontier for automating cross-hardware optimizations.