ON1 (G116 V8): 38μs Black-Box AI Memory Retrieval on Virtual Chip ISA (github.com)

ON1 has unveiled the G116 v8, a groundbreaking virtual chip architecture that introduces a transparent memory retrieval system for AI applications, achieving an impressive latency of just 38 microseconds. Unlike traditional chips that obscure latency metrics behind a single query time, the G116 v8 breaks down vector retrieval into three observable stages: the Fetch Layer (0.1-0.5 μs per operation), the Compute Layer (0.4-2 μs), and the Search Layer (3-10 ms for brute-force), making it a more intuitive tool for developers working with large language models (LLMs). This innovative approach is significant for the AI/ML community as it addresses critical bottlenecks often encountered in real-time retrieval-augmented generation (RAG) tasks. The G116 v8’s architecture not only improves the efficiency of memory access and computation but also anticipates the coming integration of advanced indexing solutions like FAISS and GPU acceleration. Moreover, with a live public verification endpoint available, users can directly measure and validate the latency decomposition, offering a new, transparent perspective on AI processing that could enhance applications involving complex natural language grounding.