Tool search should not be search but discovery (nicolaygerold.com)

Anthropic has introduced a new feature called tool_search, aimed at enhancing the interaction between large language models (LLMs) and tools. This feature is designed to defer the loading of tool definitions until they are necessary, theoretically optimizing context usage in LLM prompts. However, the implementation has faced challenges, particularly around the model's ability to recognize and utilize available tools. Users are reporting issues with the model ignoring deferred tools and struggling to differentiate between tools with similar functions, indicating a need for more effective methods of tool discovery. The significance of tool_search lies in its potential to streamline how LLMs access and use tools, reducing the burden of context token overload. Currently, many users find that using multiple tools can quickly consume a large percentage of their context window. The proposed solutions involve creating a more intuitive way for models to discover the tools at their disposal, moving beyond simple search mechanisms to a discovery framework that informs the model about available tools. This could lead to more effective tool utilization and improved LLM performance in handling complex tasks.