Show HN: Jiwo – 0.8B/4B decision models for low latency scenarios (github.com)

Jiwo has unveiled two new decision models, jiwo-0.8b and jiwo-4b, optimized for low-latency applications, achieving median request times of approximately 50 milliseconds on an NVIDIA H100 GPU. Unlike traditional models, Jiwo does not generate text but processes state and typed questions to deliver probabilities for each option in a single forward pass. Jiwo-0.8b leads the decision index for models under 1 billion parameters, while jiwo-4b ranks first in the 3-6 billion category, underscoring its competitive edge in efficiency and performance. The significance of Jiwo lies in its tailored approach to decision-making tasks, providing a robust alternative for applications that require swift and reliable probabilistic assessments. Technical innovation within the models ensures they can handle requests of up to 64 questions, offering scores of confidence for decision-making without the overhead of text generation. This positions Jiwo as a valuable tool for developers seeking to enrich applications with rapid and context-sensitive decision capabilities, especially in customer service and automated response systems. With easy integration through a REST API, developers can swiftly adopt these models for real-world applications, reinforcing the growing trend of deploying AI/ML solutions in high-speed environments.