Show HN: EvalView – Catch agent regressions before you ship (pytest for agents) (github.com)

EvalView has launched an open-source testing framework specifically designed for AI agents, akin to pytest, which allows developers to create readable test cases and integrate them into CI/CD pipelines. The framework supports multiple platforms, including LangGraph, CrewAI, and tools from OpenAI and Anthropic, thereby broadening its applicability across different AI applications. With EvalView, developers can automate regression testing to ensure that behavior, tool calls, cost, and latency do not degrade over time, which is critical for maintaining high-quality AI applications. This framework is significant for the AI/ML community as it automates testing processes that are often manual and time-consuming, allowing for more immediate feedback on agent performance and the detection of issues like hallucinations directly in production environments. Key features include YAML-defined test cases, real-time evaluation of tool accuracy and output quality with LLM-as-judge capabilities, and comprehensive reporting of performance metrics, making it easier for developers to pinpoint regressions and optimize their AI systems. Additionally, EvalView's ability to auto-generate tests from real interactions significantly enhances the testing process, streamlining workflows for solo developers and small teams.