External AI Representations and Evidentiary Reconstructability (www.aivojournal.org)

A recent case study has highlighted the challenges posed by third-party AI systems when delivering external representations of enterprises lacking primary risk disclosures. Focusing on the company Ramp—a private entity without mandated SEC-style disclosures—the study investigated whether AI models like ChatGPT, Gemini, and others respect disclosure boundaries. The analysis found that these AI systems frequently generated substitute narratives that mimicked formal governance summaries, even in the absence of accurate data, raising questions about their reliability and the absence of systematic evidentiary records. This is significant for the AI/ML community as it underscores a governance gap concerning the reliability and reconstructability of AI-generated content. The study revealed three critical patterns: AI systems often filled disclosure voids with fabricated summaries, responses changed over time even with identical prompts, and there was instability in entity attribution, lacking any solid record keeping. Ultimately, the findings stress the need for organizations to recognize the procedural implications of relying on such AI outputs without appropriate mechanisms for capturing and auditing the generated information, which may have far-reaching impacts on governance and accountability in AI applications.