Sub-15ms, non-autoregressive, local drop-in alternative to TypeSafe Jev (github.com)

The recent announcement of Von-1.0, an open-source non-autoregressive model, presents a significant advancement in decision-making efficiency for the AI/ML community. Unlike conventional autoregressive large language models that output data token-by-token—resulting in latency ranging from 500 to 2,000 ms—Von-1.0 operates in under 15 ms for local processing, using a single forward pass to evaluate multiple discrete and continuous criteria simultaneously. This paradigm shift not only enhances speed but also minimizes key-value memory overhead and eliminates nondeterministic parsing errors, making it particularly suitable for tasks like classification and intent routing. Von-1.0 boasts impressive metrics, achieving a 91.23% accuracy on adversarial multi-hop reasoning benchmarks and enhanced calibrated uncertainty through a dual loss framework. As a hardware-agnostic system, it supports native kernel optimization across various platforms, ensuring versatility and accessibility. Von is fully compatible with the existing TypeSafe protocol, positioning it as a strong alternative to proprietary models like TypeSafe Jev, which operates in the cloud at higher latency and cost. This open-source approach empowers developers to integrate advanced decision-making capabilities into their applications, fueling innovation across diverse sectors.