Same model, different agent: 17 points apart on 21 MCP servers (m3.sineframe.com)

Recent tests compared two coding agents, OpenCode and Claude Code (both using the Claude Haiku 5.5 model), across 21 MCP servers, yielding significant performance discrepancies. OpenCode achieved an accuracy of 82.0%, while Claude Code lagged at 65.4%. The evaluation deployed M3, an open-source tool, across a substantial workload of roughly 20 tasks per server. Notably, OpenCode excelled particularly on servers requiring built-in tools, underscoring the impact of agent capabilities on task completion. This study holds critical implications for the AI/ML community, particularly in understanding how agents interface with various server configurations. The performance gap highlights that different agents interact with servers in distinct ways, often bypassing the intended source for answers. For developers deploying MCP servers, the results illustrate the importance of choosing the right testing agent to accurately gauge server performance, as an agent’s inherent tools can skew success rates. Consequently, it may be essential for users to familiarize themselves with the operational nuances of their agents to optimize server task success.