What I'd audit on an AI-built SaaS before its first paying customer (knbrlo.com)

A recent audit revealed significant vulnerabilities in AI-built software-as-a-service (SaaS) applications, particularly concerning tenant data isolation. After launching a minimum viable product (MVP), one customer was able to access another's data due to a missing tenant-isolation check in the code, highlighting the fragile nature of behavioral checks when AI generates code at scale. This incident underscored a broader issue within the AI/ML community: reliance on surface-level controls without enforcing strict code structures can lead to severe data breaches when real customers come on board. To address these pitfalls, the article proposes a rigorous audit framework for AI-built SaaS, emphasizing structural fixes over behavioral ones. Key recommendations include implementing authorization as a type at the boundary, utilizing database-level security measures to ensure strict tenant isolation, and establishing robust evaluation processes rather than traditional unit tests for AI model outputs. Furthermore, the article advocates for real-time cost attribution per tenant for model usage, allowing for better financial management and transparency. The insights are crucial for developers and teams creating AI-driven software, aiming to bolster security and reliability in production environments before their first paying customers come onboard.