Show HN: Vyne, a 205MB on-device decision model with typed, calibrated outputs (vyne.toolchain.studio)

Vyne has introduced a new on-device decision model, featuring a compact size of just 205MB while providing typed and calibrated outputs. This innovative model consists of a full-precision build and a ternary version, critically designed to achieve performance akin to larger models while minimizing resource consumption. In independent evaluations, Vyne demonstrates notable advantages over the TypeSafe JevAPI, outperforming it on several tests, including the CLINC150 benchmark, where it secured a 96.3% accuracy rate compared to Jev's 87.0%. Additionally, Vyne significantly reduces calibration error, achieving a four-fold improvement. The significance of Vyne lies in its balance of performance and efficiency, making it well-suited for deployment on edge devices where computational resources are limited. By integrating ternary weights and 8-bit embeddings, Vyne optimizes its operational footprint without compromising too much on accuracy, opening avenues for more widespread AI applications in mobile and IoT environments. This advancement has implications for both developers and end-users, enhancing the feasibility of implementing sophisticated machine learning models in everyday devices while minimizing the overhead associated with larger models.