The State of AI: don't share your secrets with a chatbot (www.ft.com)

New guidance: don’t treat chatbots like secure vaults. Recent coverage warns users and organizations that conversational AI systems routinely retain, analyze, and sometimes expose sensitive input. Beyond obvious privacy-policy concerns, the AI/ML threat surface includes model-inversion and membership-inference attacks that can reconstruct training data, prompt-injection exploits that force models to reveal secrets (API keys, proprietary prompts), and careless logging or fine-tuning pipelines that persist user content. Because large language models process everything in plain-text context windows and many vendor APIs store transcripts to improve models, casual sharing can leak intellectual property, personal data, or regulated information. This matters because chatbots are rapidly integrated into workflows and customer-facing tools, extending the risk to enterprises and developers. Technical mitigations include on-prem or private-instance deployments, strict data retention and logging policies, input/output redaction, differential privacy or federated learning for model updates, and adversarial testing (red-teaming) to find prompt-injection paths. Operational controls — role-based access, rate limits, encryption-in-transit and at-rest, and careful RLHF training data curation — are equally important. The takeaway: treat LLMs as powerful but porous tools; assume anything you submit might be used for model improvement or exposed, and design systems and policies accordingly.