Credence – Jev-style typed decisions from a local GGUF model (github.com)

Credence has emerged as an innovative local inference runtime designed for making typed probabilistic decisions utilizing GGUF language models. Distinct from conventional agent frameworks, Credence provides a streamlined approach where decisions are derived from the model's next-token distribution focused on a predetermined set of labels. This results in efficient decision-making with minimal operational cost—only requiring one prompt decode to yield a boolean outcome, alongside crucial diagnostics that assess the reliability of probability judgments. The significance of Credence for the AI/ML community lies in its ability to enhance decision-making processes by embedding probabilistic models locally, thereby reducing reliance on cloud-based systems and improving privacy. The ongoing Phase 1 features a fully operational model-backed decision path and robust profiling capabilities, including Platt calibration for more refined predictions. With built-in support for configurations on various operating systems and a detailed evaluation set, researchers and developers can leverage Credence to fine-tune model performance and enhance accuracy in probabilistic assessments. The project is open-source under the Apache License 2.0, fostering collaborative development and broader usage across diverse applications in AI.