Leaky Language Models: Stealing Architecture and Inference Optimizations (arxiv.org)

Researchers have unveiled "Leaky Language Models" (LeakyLMs), a novel set of attacks capable of extracting proprietary information from production language models through an analysis of token generation timing. This marks a significant advancement in understanding the vulnerabilities of AI systems, as the method can glean critical details about architecture and inference optimizations without direct access to the models, even when interfacing with remote APIs. Notably, the attacks identify whether providers use techniques like speculative decoding and reveal specific contextual parameters, as evidenced by findings regarding Google’s Gemini Flash 2.5. The implications of LeakyLMs are profound for the AI/ML community, highlighting potential security risks in deploying language models. By accurately modeling timing on modern NVIDIA GPUs, the research demonstrates that details such as transformer layers and attention heads can be inferred with high accuracy—over 90% of the time, the correct architectural configuration was among the top ten guesses in experiments. This work serves as a wake-up call to developers and organizations using language models, emphasizing the need for robust security measures to safeguard against the unintended leakage of sensitive model details.