Show HN: OSS Tests to Fix AI Gen Code. 110 Test for Major API – Supabase, Auth0 (github.com)

A new open-source tool called api-doctor has been launched to enhance the reliability of AI-generated code by scanning for issues related to API integrations. Utilizing deterministic Abstract Syntax Tree (AST) rules, api-doctor ensures consistent outputs regardless of input variations, distinguishing it from typical AI models that operate on prompts. The tool supports major API providers such as Supabase, Auth0, and Firebase, with each integration being assessed against 110 specific rules categorized into security, correctness, reliability, and integration. This addresses common issues like hardcoded API keys, incorrect endpoints, and production failure modes, which can significantly impact software performance. This development is significant for the AI/ML community as it offers a solution to a crucial challenge: the inability to effectively test AI-generated code without introducing additional complexity. By breaking the feedback loop where AI is used to validate AI, api-doctor promotes greater trust and accuracy in integrating AI within applications. Additionally, the tool respects user privacy by collecting only necessary anonymized usage data, thus providing meaningful insights without compromising sensitive information. This initiative highlights a growing recognition of the need for robust testing frameworks in the evolving landscape of AI-driven development.