Show HN: Eval-skills for automatic agent improvement (github.com)

A new tool, Eval-skills, has been introduced to enhance the capabilities of coding agents in improving AI applications through structured evaluation and feedback. This toolkit allows developers to build custom evaluations ("evals") that assist in error analysis, validation of graders, and testing. Significantly, Eval-skills supports integration with popular coding assistants like Claude Code and Codex, enabling users to leverage their existing datasets and evaluation techniques rather than starting from scratch. The focus is on real application executions, promoting a more intuitive understanding of AI behavior over abstract metrics. This framework is vital for the AI/ML community as it streamlines the testing and improvement process of AI products. It emphasizes hands-on engagement with application performance, offering workflows for engineers, product managers, and individual developers alike to analyze errors, group them by failure modes, and create trusted evaluation metrics. Users can follow a bounded experimentation process known as "Descent," which facilitates incremental improvements while protecting against regressions. Overall, Eval-skills aims to foster a more collaborative and efficient environment for AI developers seeking to refine their technology through actionable insights and validated evaluation practices.