Practical Secrets Extraction Against Black-Box LLMs (arxiv.org)

Researchers have introduced a novel black-box secret extraction framework targeting large language models (LLMs) that are accessed via APIs, such as OpenAI’s Codex and Claude Code. This method addresses the significant risk of sensitive information leakage, as LLMs may inadvertently memorize confidential data from their training corpora. The framework employs two key techniques: Cross-Validated Secret Knowledge Distillation, which uses semantic prompts and response validation to create a local proxy of the model, and Proxy-Guided Secret Extraction, which utilizes advanced sampling and profiling methods to recover hidden secrets. This development is crucial for the AI/ML community as it highlights vulnerabilities in widely-used LLMs, raising awareness about the potential breaches of private information. The framework demonstrates improved extraction effectiveness on benchmarked API keys over existing methods, reducing both latency and recovery time. By successfully uncovering masked credentials from several deployed LLM systems, this research emphasizes the urgent need for enhanced security measures in the deployment and usage of AI language models, promoting a more responsible and secure AI landscape.