Mock LLM APIs locally with real-world streaming physics (vidai.uk)

Vidai.Server has launched VidaiMock, a groundbreaking open-source tool designed to facilitate the local simulation of large language model (LLM) APIs with real-world streaming physics. Unlike traditional mocks that cater to REST APIs, VidaiMock is tailored to emulate the probabilistic behavior of LLMs, enabling developers to simulate conditions like network jitter, latency, and failure modes without relying on heavy runtimes or containers. This innovative tool promises to enhance testing suites with features like real-time token simulation and automatic configuration for popular providers such as OpenAI and Anthropic. The significance of VidaiMock for the AI/ML community lies in its ability to provide a zero-cost, highly efficient local testing environment. As developers increasingly seek to assess and improve the resilience of their applications under various conditions, VidaiMock’s capabilities—such as injecting rate limits, TCP timeouts, and advanced Retrieval Augmented Generation mocking—deliver crucial insights into production-level behavior. With its lightweight Rust binary and built-in observability features, VidaiMock represents a substantial step forward in creating more sophisticated, scalable AI applications while reducing setup time and resource consumption.