Brio mode in Colibri: Scoring a closed set instead of generating (github.com)

Colibri has introduced Brio mode, a significant enhancement that shifts the focus from generating responses to scoring options within a closed set. In this mode, users provide a prompt along with predefined choices, and the model evaluates how likely each option is, accompanied by an entropy value that reflects its confidence. This approach addresses critical decision-making questions, providing insights into which options are the most probable and how certain the model is about those probabilities. Unlike traditional generative models, which can only produce strings without an indication of confidence, Brio mode quantifies certainty, thereby enabling automated decision processes that were previously unattainable. Brio mode serves through various interfaces, including an HTTP endpoint and a terminal command. Its architecture emphasizes efficiency by utilizing a shared memory snapshot for context, ensuring that subsequent queries are processed quickly without repeated loading overhead. The model's ability to present entropy levels also allows users to gauge when human intervention may be necessary, as higher entropy indicates uncertainty. These features optimize decision-making workflows, especially in tasks requiring precise judgments, such as code reviews. Overall, Brio mode marks a leap forward for AI/ML applications, enhancing the reliability and utility of AI in critical decision-making scenarios.