Show HN: Flide: Make Jev-like decision models up to 9.5x faster (github.com)

Flide has launched its fast inference infrastructure for Jev-style decision models, achieving speeds up to 9.4 times faster than existing frameworks like vLLM and SGLang. The latest version, Flide 0.1.9, enhances functionality by allowing multiple requests in one API call, significantly improving throughput. The platform supports various decision model families, including the newly added pplx-decider-v1.1-27b and JEV-35B, providing rapid responses while maintaining high fidelity with just a 1.1-point difference from reference models. This development is pivotal for the AI/ML community as it streamlines decision-making processes in applications ranging from customer support to automated systems. Flide utilizes exact state forking, enabling models to operate efficiently with reduced computational overhead while still delivering accurate answers. The introduction of features like batch-invariant requests allows for consistent performance regardless of input size, enhancing user experience in real-world applications. By supporting multiple model types and expanding its infrastructure, Flide positions itself as a critical tool for developers aiming to integrate fast, reliable decision-making capabilities into their AI solutions.