QwenPU: An LLM running inside another LLM's weights (huggingface.co)

QwenPU is a groundbreaking development in the AI/ML landscape, enabling an LLM (Large Language Model) to function as a virtual computer. It integrates a 32-bit RISC-V CPU and a vector unit directly into the weights of Qwen3.5-4B, allowing the model to execute programs, such as Andrej Karpathy's stories260K—a 260K-parameter Llama model—during its forward pass. This technology signifies a shift in how LLMs can leverage computational resources, effectively allowing them to perform complex operations internally rather than solely relying on external processing. With a checkpoint size of 26.7 GB, QwenPU is designed to operate on GPUs of at least 40 GB capacity, highlighting the significant computational demands of this architecture. Each vector operation within QwenPU executes roughly 3,300 trace tokens per second, markedly improving processing times compared to its predecessor, Qwenputer-npu. The model can run a full story program with zero mismatches, streamlining the generation of content while maintaining high fidelity to the input data. The implications of QwenPU extend beyond mere performance metrics; it opens new avenues for efficient model design and integration within AI systems by embedding computational engines in LLM architectures.