I let an LLM rewrite the tool descriptions of 4 MCP servers – before/after data (toolmetry-one.vercel.app)

A recent experiment involved using a large language model (LLM) to rewrite tool descriptions for four popular MCP servers, aiming to improve agent utilization. By analyzing the original and revised descriptions, the study sought to quantify improvements in accuracy and efficiency when agents interact with the servers. The results were striking: the official SQLite server's successful tool calls jumped from 34% to 100%, with other servers also showing substantial gains. These improvements were achieved with minimal costs and without modifying the servers themselves, showcasing an efficient solution for enhancing agent performance. The significance of this study for the AI/ML community lies in highlighting the critical role of clear tool descriptions in optimizing agent interactions. The findings reveal that weaker models are heavily impacted by poor descriptions, while stronger models can often bypass these issues. The experiment also stresses the importance of iterative testing, as LLM rewrites can yield variable results, necessitating a careful measurement approach to ensure reliability. This research not only provides actionable insights for improving operational performance in MCP environments but also emphasizes that thoughtful optimization of agent interfaces can be both cost-effective and easily reversible, presenting an accessible path for developers.