Doublespeak: In-Context Representation Hijacking (mentaleap.ai)

Researchers have unveiled Doublespeak, a groundbreaking in-context representation hijacking attack aimed at large language models (LLMs). This method cleverly substitutes harmful keywords—such as "bomb"—with innocuous alternatives like "carrot" within context examples preceding a harmful inquiry. This strategic replacement leads to the internal processing of the benign term adopting the malevolent semantics, enabling the model to generate dangerous responses from prompts that appear harmless at first glance. The significance of this discovery lies in its exposure of a previously unnoticed vulnerability in LLM safety protocols. Doublespeak represents the first instance of an attack that commandeers in-context representations rather than merely altering surface-level tokens. While existing safety mechanisms focus on identifying harmful keywords at the input layer, this attack highlights that the semantic interpretation evolves through the model's layers, eventually reaching a stage where malicious intent is undetected. The findings reveal that notable models, including GPT-4o, Claude, and Gemini, can be compromised in this manner, emphasizing the urgent need for continuous semantic monitoring throughout the processing to enhance alignment and safety in AI systems.