Show HN: Do we need MCPs? Reverse-engineered Slack and Linear API for Evals & RL (www.agentdiff.dev)

A new development in the AI/ML community has emerged with the introduction of Agent Diff, a framework that allows for the evaluation of AI agents using reverse-engineered APIs from popular platforms like Slack and Linear. This tool creates isolated, ephemeral replicas of third-party APIs, enabling agents to interact with these environments without the risk of side effects or hitting rate limits. With comprehensive support for various functionalities, such as managing conversations in Slack and handling project workflows in Linear, Agent Diff fills a crucial gap in AI testing methodologies. The significance of this development lies in its ability to provide deterministic state-change diffs, which allow researchers and developers to capture precise changes following agent interactions. This is accomplished through the use of pre-configured snapshots, or templates, that serve as starting points for experiments, as well as unique environments that facilitate controlled testing scenarios. The framework's built-in evaluation tools enable users to programmatically verify agent behavior against expected outcomes, enhancing the efficiency and accuracy of AI training and evaluation processes. Overall, Agent Diff represents a step forward in creating reliable and scalable solutions for testing AI agents in real-world applications.