Mini-Jev – typesafe's Jev implemented on top of an LLM locally (github.com)

TypeSafe has introduced Mini-Jev, an innovative approach that utilizes a Jev-style interface on a frozen AI model, specifically the Qwen3-4B. Rather than generating JSON outputs for typed decisions, Mini-Jev classifies input based on a provided JSON schema by reading next-token logits directly, turning multiple-choice questions into lettered fields and running a single forward pass through the model. This method, tested against the CLINC150 dataset, demonstrated that it matches the accuracy of traditional JSON generation while being significantly faster—up to four times quicker for shorter texts. This development holds substantial implications for the AI/ML community by suggesting that many frozen models inherently support efficient decision-making processes without extensive generative tasks. The findings indicate that using lettered options maintained accuracy comparable to JSON outputs, while also minimizing the decoding costs associated with generating strings. The study emphasizes the potential for improved performance in AI applications requiring quick, typed responses, paving the way for new design philosophies that prioritize efficiency in AI interactions.