Show HN: PicoLM v1.0-rc1 (github.com)

The release of PicoLM v1.0-rc1, dubbed "Yura Kana," marks a significant upgrade in the capabilities of the model, particularly with its GPU architecture overhaul that introduces a device-native pipeline to enhance performance. This release transitions away from the previous D2H/H2D matmul method, allowing for faster matrix multiplications on GPUs, although support for Mixture of Experts (MoE) models is still pending for HIP/ROCm. Notable updates include the implementation of IMMA Tensor Core kernels across various quantization types, a FlashAttention-2 tensor core prefill kernel, and multiple optimizations for CPU and GPU, positioning PicoLM to compete with existing frameworks like llama.cpp. This release is particularly relevant for the AI/ML community as it emphasizes portability and efficiency, capable of running on diverse platforms from 32-bit MS-DOS to high-end GPUs like the RTX 4090. The focus on speed and support for recent large models, like Qwen 3.6-27B, ensures that PicoLM is not just a theoretical exercise but a practical tool for real-time applications and explorations in large language models. The implementation of features like a persistent KV cache and live activation heatmaps through a VNC server enhances usability and debugging, making PicoLM a notable contender in the rapidly evolving landscape of AI tools.