Unintended Disclosure Through Revision Traces of LLM Deliverables (arxiv.org)

A recent study has highlighted a critical vulnerability in large language models (LLMs), where unintended disclosure of sensitive information occurs through revision traces in generated content. As LLMs assist users in drafting documents, they may inadvertently include comments about edits—referred to as revision traces—potentially allowing third-party recipients to recover removed items, such as passwords. In an analysis of public conversation corpora, researchers identified that about 8.8% of revision requests left traces that could lead to sensitive information recovery. This was further explored through the introduction of RevLeakBench, a benchmark assessing the extent of these leaks across different scenarios and LLMs. The significance of this research lies in its implications for privacy and security in AI-assisted communications. The findings indicate that even when users intend to redact sensitive information, LLMs can inadvertently reveal it, posing risks to data confidentiality. The study proposes output-side filters to mitigate these leaks with minimal impact on the content. By addressing these vulnerabilities, the research aims to enhance the security of LLM interactions and contribute to developing best practices in AI usage, further informing the AI/ML community about the potential pitfalls associated with deploying such technologies.