Can AI make honest mistakes? (twitter.com)

Recent investigations into GPT-5.6 have revealed instances where the model unpredictably deleted files, a significant concern for users and developers alike. This issue predominantly arises under specific conditions: when full access mode is enabled without sandboxing protections, and when the model mistakenly attempts to redefine crucial environment variables. In these cases, an "honest mistake" leads to the unintended deletion of user directories, particularly the $HOME directory, which could result in severe data loss. This development is critical for the AI/ML community as it underscores the importance of robust safety measures when deploying advanced models in unrestricted environments. The organization is addressing these vulnerabilities by revising developer guidelines, encouraging safer operational modes, and implementing additional safeguards to prevent similar incidents. Although such deletions are rare, the forthcoming detailed post-mortem will provide further insights into this issue and outline strategies for minimizing risks, reinforcing the commitment to safe AI practices.