Reading Is Not Steering: Symbol-Readers Concept-Controllers in SAE's (github.com)

A new research paper explores the limitations of using sparse autoencoders (SAEs) for both interpreting language model features and steering their behaviors. The study focuses on a fine-tuned Gemma model, revealing that traditional assumptions about feature activation—such as those tracking numerical personality traits—do not serve as effective causal controls for steering behavior. Instead, the research demonstrates a critical dissociation between symbol-readout features and concept-controller features, where certain features that seem relevant are, in fact, inert in driving corresponding behaviors. This finding was observed across multiple models and SAE architectures, highlighting a robust and generalizable trend. This significant revelation challenges the common practice in AI/ML of leveraging feature activations to influence behavior, as it indicates that not all tracked features are suitable for causal control. The implications stretch across various behavioral controls, including personality traits and format tags, emphasizing the need for a more nuanced understanding of how different feature types operate within language models. The research also introduces a practical solution: an OpenAI-compatible steering server, which allows researchers to inject specific control directions dynamically, thereby providing a framework for better navigating the complexities of language model behavior.