Trustbut checks if an AI agent's new tests fail without its change (github.com)

Trustbut has introduced a new tool that serves as an independent verification system for code changes made by AI agents. It effectively checks whether the tests pass during a clean checkout, ensuring that any changes made to the code do not introduce failures or skip necessary tests. The tool generates a "receipt" for pull requests, allowing developers to verify the integrity of the code irrespective of who or what wrote it—human or AI. By integrating directly into the workflow, Trustbut can halt the AI agent if any tests fail, prompting continuous improvement until all changes are properly validated. This development is particularly significant for the AI/ML community as it addresses the critical issue of trust in AI-generated code. With a substantial number of developers expressing concerns over AI reliability, Trustbut aims to bridge the trust gap, providing concrete evidence that validates AI outputs. It supports various programming languages and test frameworks, making it accessible and practical for diverse projects. By enforcing rigorous testing and accountability in AI-assisted coding, Trustbut is poised to enhance confidence in AI tools and their outputs.