Show HN: Evolved x86 AVX-512 kernels for NF4 LLM inference (github.com)

A new AI tool called Genesis has been introduced, focusing on optimizing local inference for NF4-quantized Large Language Models (LLMs) using AVX-512 instruction sets. By fusing weight dequantization and matrix multiplication into a single operation, Genesis significantly enhances performance, achieving a latency of just 0.15 ms per expert compared to 24.8 ms using traditional methods. This innovative approach is particularly effective for Mixture of Experts (MoE) models that require CPU offloading, now allowing large models like the Qwen3-Next-80B-A3B to run efficiently on a single GPU without materializing the extensive weight matrices. The significance of Genesis lies in its evolutionary design process, which utilized genetic algorithms to determine the optimal order of x86 instructions based on hardware evaluations. This method led to kernels that outperform hand-tuned alternatives, leveraging the specific architectural features of AMD's Zen 4 processors. The integration of AI to evolve performance-enhancing kernels represents a significant step forward in both practical application and research methodologies in AI/ML, providing a potent tool for developers dealing with resource-intensive language models. The code is open-source, facilitating community usage and further advancements in the field.