Sure, Your Claude Is Doing Things. Prove It (runme.dev)

Runme has announced the release of runme eval (v3.17), a new tool designed to enhance the testing of AI workflows beyond traditional methods. The tool addresses the pressing issue that while agent skills (like those used in Claude, ChatGPT, or Codex) appear functional, they often lack rigorous testing similar to software code. With runme eval, developers can implement systematic evaluations—such as smoke tests and integration tests—on workflow components to ensure their continued functionality as environments change. This development underscores a significant shift in the AI/ML community towards treating AI workflows with the same scrutiny as software, validating skills not just based on luck or anecdotal evidence but through repeatable, evidence-driven assessments. This advance is vital for maintaining the reliability of AI applications, addressing a critical gap in how developer workflows are currently validated. The tool enables users to structure evaluations around specific tasks or skills, capturing evidence and performance metrics that inform ongoing improvements. By creating a structured framework for assessing agent behavior, runme eval lays the groundwork for more robust machine learning applications, allowing teams to focus on refining their workflows while gaining confidence in the systems they deploy. This shift towards a discipline centered on empirical testing rather than informal assessments could lead to greater trust in AI models across industries.