Joint Optimization of Tool Creation and Use for Large Language Model Agents (tool-use-smith.github.io)

Researchers have introduced SMITH (Schema-grounded Multi-task Iterative Tool Honing), a novel reinforcement learning framework designed to optimize both the creation and usage of tools in larger language model (LLM) agents. Unlike previous methods where a powerful model creates tools and a separate weaker model utilizes them, SMITH integrates these tasks into a single learning policy. This joint training enables immediate feedback for the tool creator, improving the accuracy of generated schemas and tool functions by allowing gradients from tool usage to inform the creation process. A model trained using SMITH demonstrated significant improvements in accuracy across various procedural reasoning tasks, achieving 79.9% macro-average accuracy on held-out tasks, outperforming larger models with untrained tool writers. The significance of SMITH lies in addressing a fundamental limitation in tool-augmented LLMs—existing tools may not meet the needs of specific tasks, leaving agents ineffective without the right tools. By incentivizing successful tool creation through direct feedback on usage, SMITH addresses the dual challenges of reward decomposition (identifying failures in schema, code, and design) and circular evaluation (ensuring that tool evaluation improves alongside model updates). As a result, SMITH not only enhances the model's ability to create adaptable, reusable tools but also encourages efficient resource use, processing far fewer tokens than traditional methods while maintaining superior performance in unseen domains.