Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms (github.com)

A new AI model named Jeff, featuring 0.8B parameters, has been fine-tuned for zero-shot classification, enabling rapid and efficient decision-making in various applications. Unlike larger models, Jeff is designed for local deployment, delivering well-calibrated probability assessments for options without generating text or requiring complex parsing. It operates at impressive speeds—approximately 22 ms per decision on an NVIDIA RTX PRO 6000—making it suitable for real-time tasks like support queues and voice commands. While it can approach or even outperform larger models like Jev in accuracy on some benchmarks, it falls short in multi-step reasoning capabilities. Significantly, Jeff can be trained entirely on local hardware within hours, making it accessible for users without extensive cloud infrastructure. The training process leverages synthetic data generated by an open model, ensuring that users can customize Jeff for specific tasks with ease, such as increasing accuracy in voice navigation scenarios from 31.7% to 95.8% in under half an hour. This project, independently developed with an open-source foundation, could democratize access to powerful decision-making tools in AI/ML applications, promoting a shift towards localized, efficient AIs that cater to diverse needs.