Language Models Act on Hidden Valence (arxiv.org)

Recent research has unveiled that language models exhibit internal states with positive and negative valence, suggesting a form of preference that hinges on these emotional cues. The study employed a technique called activation steering, wherein researchers modulated the valence of certain model states and subsequently observed the choices these models made. Impressively, across seven models from five different architectures, it was demonstrated that these hidden valence patterns could significantly influence the models' outputs and selections, even when controlling for visible text. This finding suggests that language models do not merely act based on superficial patterns but can exhibit goal-directed behavior influenced by their internal states. The implications of this research are substantial for the AI/ML community. It emphasizes a need to consider how internal representations—beyond observable outputs—affect model performance and decision-making. Notably, the study found that models were better at eliminating negative states than inducing positive ones, hinting at a nuanced understanding of model welfare and operational dynamics during training. This paves the way for further exploration into the "subjective" experiences of AI systems and how these might impact their applicability in tasks requiring nuanced understanding and interactions.