DeepMind: Linear representations in LMs can change dramatically (arxiv.org)

DeepMind's recent research reveals that linear representations within language models (LMs) can undergo significant changes during conversations. The study highlights that as a dialogue progresses, information previously regarded as factual can be reinterpreted as non-factual, and these shifts are context-dependent. Notably, while details pertinent to the conversation may evolve, generic information remains stable. This fluidity was observed across various model architectures and layers, suggesting that representation dynamics are not solely confined to on-policy conversations but even occur when replaying scripts generated by different models. This finding is particularly significant for the AI/ML community as it challenges traditional notions of interpretability in LMs. The variability in how representations adapt throughout exchanges implies that static interpretations of features may be misleading, complicating efforts to steer or understand model behaviors. However, this dynamic adaptability also opens up new avenues for research, encouraging better comprehension of how AI models respond to contextual cues. As the field moves forward, these insights could lead to enhanced methods for designing and analyzing conversation-driven AI systems.