Show HN: Decido – an open decision model that runs CPU-only on a laptop (github.com)

Decido has launched an innovative open decision model capable of running on standard laptop CPUs, enabling users to make calibrated yes/no, pick-one, and rating decisions about documents. By leveraging a LoRA fine-tune of the Gemma 4 26B-A4B model with approximately 4 billion active parameters, Decido outputs a probability for each answer and is designed to be compatible with TypeSafe's Jev "System One" API. The model runs efficiently on devices such as the Apple M4 Pro, processing a 1,000-token document with five questions in just 4.1 seconds, a significant achievement given the latency challenges typically associated with complex AI models. This development is particularly noteworthy for the AI/ML community as it democratizes access to advanced decision-making capabilities without the need for high-powered GPU hardware. The open-source nature of Decido allows researchers and developers to explore and enhance decision-making processes in various applications, from customer service interactions to data analysis. Furthermore, with the model's architecture supporting up to 16,000 tokens in requests, it opens new avenues for handling extensive datasets in a CPU-focused environment, potentially increasing the adoption of AI in settings with limited resources.