Why CUDA translation wont unlock AMD (eliovp.com)

Many vendors and projects pitch “CUDA translation” tools that let you compile existing CUDA code for AMD GPUs with no source rewrite, promising “native” performance. In practice those layers — nvcc-compatible frontends, PTX parsing, LLVM backends, and CUDA-X→ROCm wrapper libraries — can get legacy workloads running, but they hit a hard ceiling for cutting‑edge AI. CUDA was designed around NVIDIA semantics (warp=32, PTX intrinsics, CUDA-X tuned kernels); AMD Instinct (CDNA) uses wave64, different memory/cache tradeoffs, MFMA/GEMM shapes, and FP8 formats (E4M3/E5M2) with distinct packing/scaling and KV‑cache behaviors. Straight translation often yields masked lanes, suboptimal occupancy, missed fused kernels, and fallback FP8 paths — meaning the code “compiles” but can be ~2× off peak or fail to use AMD FP8 fast paths. That gap matters: modern LLM throughput depends on fused kernels, AMD‑tuned GEMMs, precise FP8 flow, and topology-aware tensor parallelism — things a generic translator won’t invent unless it reimplements AMD‑first engineering and perpetually chases ROCm releases. The practical takeaway: translators are useful for quick porting or legacy/HPC code and consumer RDNA (wave32) experimentation, but to extract MI‑class performance you need ROCm/HIP native stacks, vendor‑tuned libraries, and AMD‑specific kernels. Projects like Paiton demonstrate this AMD‑first approach — ROCm integration, custom kernels, FP8 optimizations, and topology tuning — delivering real MI300X LLM gains where translation layers fall short.