Show HN: We created API-Bench to test how well LLMs execute against APIs (superglue.ai)

API-Bench, now in its second version, has been introduced as a rigorous tool for evaluating how effectively large language models (LLMs) can build integrations with APIs. This benchmark specifically assesses whether LLMs can execute multi-step workflows against real-world APIs, covering aspects like authentication, adherence to API specifications, and error recovery. The key takeaway is that while LLMs demonstrate proficiency in generating code, they often lack the reliability needed for seamless integrations, which is crucial for production environments. The findings reveal significant performance gaps among various models, with the specialized integration layer superglue achieving a 93% success rate compared to lower rates for popular LLMs like GPT-5 and Claude Sonnet 4.5. Moreover, common pitfalls include outdated training on API documentation, difficulties in debugging and managing authentication, and challenges with less prevalent systems. These insights highlight the importance of robust documentation and error-handling capabilities, suggesting that successful adoption of AI agents will depend not only on their reasoning abilities but also on their functional reliability in integration tasks. API-Bench is open-source, enabling the community to run tests and contribute to the evaluation of API integration frameworks.