Show HN: I wrote an LLM inference engine in pure Go – 48 tok/s zero dependencies (github.com)

A new inference engine for large language models (LLMs) has been developed entirely in Go, making it notable for its zero dependencies beyond the standard library. This engine, which allows users to load GGUF models and execute models efficiently on CPUs, offers features such as text generation, multi-turn chat, and real-time streaming. With impressive throughput rates of 25+ token formats supported, the engine achieves around 31 tokens per second for popular models like LLaMA 3.2 and up to 16 tokens per second for more complex models like Qwen3.5. This development is significant for the AI/ML community as it demonstrates the potential of Go in deep learning applications, which has been less explored compared to languages like Python. The implementation includes advanced features leveraging SIMD acceleration for optimized performance, and the ability to directly load quantized models without additional conversion simplifies the deployment process. This could pave the way for rapid integration of LLM capabilities into Go-based applications, enhancing performance in environments where lightweight, high-speed inference is critical.