Show HN: a Rust OS kernel built for LLM inference (github.com)

A new Rust-based operating system kernel, AXIOM, has been introduced specifically for optimizing Large Language Model (LLM) inference. This bootable, no_std kernel is not a general-purpose OS but rather an inference substrate that prioritizes inference-critical OS primitives, addressing the limitations of the traditional Linux kernel. By focusing on tensor-native memory allocation, layer-boundary scheduling, and double-buffered weight streaming, AXIOM significantly reduces streaming overhead from about 1.4 seconds to just 42 microseconds per layer. Current testing targets memory-constrained environments, particularly for 7B-class models on bare-metal NVMe storage. The significance of AXIOM lies in its tailored approach to inference workloads, which contrasts sharply with Linux's multiprogrammed workload optimization. It refines scheduling and memory management, allowing for predictable execution and caching that aligns with the specific needs of transformer networks. By removing general-purpose abstractions, AXIOM improves cache residency and minimizes latency during layer transitions, creating an environment where high-throughput inference can be achieved even under constrained resources. The kernel’s architecture is already yielding promising results, with further plans for enhancements and real-world evaluations on target hardware.