LLMs recognize dropout and Gaussian noise applied to their activations (arxiv.org)

Recent research reveals that language models can effectively recognize and localize disruptions to their activations, specifically when subjected to dropout or Gaussian noise. Tested across eight models from notable families like Gemma and Qwen with sizes ranging from 8B to 36B parameters, the models demonstrated a remarkable ability to identify perturbed sentences with high accuracy. For instance, the Qwen3-32B and Qwen3.8-27B models exhibited a correlation between accuracy and the strength of perturbation, highlighting their sensitivity to these modifications. This finding is significant for the AI/ML community as it suggests that these models possess a form of "training awareness," potentially allowing them to differentiate between various types of perturbations during inference. The implications are profound for AI safety, as understanding how these models react to such noise can inform better training techniques and enhance the robustness of AI systems. The research challenges previous assumptions about perturbations simply being less probable, emphasizing the need for further exploration into the cognitive capabilities of large language models.