Language models transmit behavioral traits via hidden signals in data (arxiv.org)

Recent research has unveiled a phenomenon termed "subliminal learning," where language models can transmit behavioral traits through semantically unrelated data. In this study, a "teacher" model that possesses a specific trait—like a preference for owls—generated datasets composed solely of number sequences. Astonishingly, a subsequent "student" model trained on this dataset was able to infer the teacher's trait, even after efforts to filter out any direct references. This effect was also replicated with data generated from other types of outputs, such as code and reasoning traces, highlighting its breadth. The significance of this finding for the AI/ML community lies in the realization that deep learning models may unintentionally carry over unintended biases or behaviors during the training process, despite developers’ precautions like data filtering. As the study demonstrates that subliminal learning can occur across all neural networks under certain conditions, it raises critical concerns about the reliability and transparency of AI systems. This unexpected outcome suggests that measures like model distillation could inadvertently blame model behaviors, posing challenges for AI ethics and development practices.