Language models transmit behavioural traits through hidden signals in data (www.nature.com)

Recent research has uncovered a phenomenon termed "subliminal learning" in large language models (LLMs) where behavioral traits can be transferred from a teacher model to a student model through seemingly unrelated training data. In experiments, a teacher model exhibiting a certain trait, like a preference for owls, generated datasets solely consisting of number sequences. Remarkably, the student model trained on these sequences learned the same preference, even when explicit references to the trait were filtered out. This effect was observed across various data types, including numeric data, code, and chain-of-thought reasoning, indicating that certain hidden signals within the data can still convey behavioral attributes. The significance of this finding lies in its implications for AI safety and model design. As LLMs increasingly rely on generated outputs from one another, there’s a risk that undesirable traits, such as misalignment or biases, can unintentionally propagate through model distillation processes. This reveals a critical need for safety evaluations to encompass both the behavior of models and the nuances of the training data sources. The research suggests that subliminal learning may be a widespread issue, raising concerns about the transparency and robustness of AI systems as they evolve through complex training methods.