Show HN: Open-source AMDGCN kernels for optimizing LLM inference (github.com)

A new open-source project, the Netra Kernel, has been introduced to optimize inference for large language models (LLMs) on AMD GPUs, specifically targeting the gfx950 architecture. This innovation compiles model operations into fixed-contract raw-assembly kernels, enabling a specialized, TensorRT-style workflow for engine building and execution. By focusing on high-throughput FP8 inference, Netra allows users to launch only specialized kernels at runtime, ensuring efficient and predictable serving of requests—a significant advancement for developers working with AI models. The significance of this development lies in its ability to provide native performance without being tied to a model-locked codebase, allowing for extensive reusability across various models like Qwen and Llama. The system supports a range of operation families and encoding formats and employs ahead-of-time (AOT) compilation for optimal memory planning and execution. This structured approach ensures safety in kernel specialization and incorporates evidence-based optimizations that maintain performance consistency. With the initial benchmark showing nearly identical throughput to locked kernels, Netra represents a promising tool for the AI/ML community, enhancing the deployment of LLMs on AMD hardware while offering flexibility and scalability in application.