Show HN: AviGPT-250M – Edge SLM with NVMe memory bus (488MB, runs offline) (github.com)

AviGPT-250M, developed by Yadlapalli Avinash Ricky in India, marks a significant advancement in edge-tier Small Language Models (SLMs) with its groundbreaking architecture that utilizes a native NVMe memory bus for enhanced performance. The model has 250 million parameters and a VRAM footprint of only 488 MB, allowing it to run efficiently offline. It sets a new standard as the world’s first SLM of its size to implement a decoupled architecture, separating parametric weights from factual storage, thus achieving a remarkable ultra-low latency of 0.002 ms for factual queries through a SQLite FTS5 engine directly interfaced with high-speed NVMe storage. Significantly, AviGPT-250M demonstrates zero parametric hallucination on indexed knowledge and 100% deterministic arithmetic accuracy, showcasing its robust capabilities in factual recall and mathematical precision. In head-to-head benchmarks against competing models, it outperforms those up to 4.4 times its size, achieving a composite efficiency score of 0.40. The model not only retains the ability to process extensive encyclopedic data swiftly but also eliminates the need for heavy vector databases, replacing them with a local SQLite engine that is significantly faster. This innovation in AI/ML represents a critical shift towards more efficient and reliable edge AI solutions.