Verbalizable Representations Form a Global Workspace in Language Models (transformer-circuits.pub)

Researchers have unveiled a significant discovery regarding the internal workings of language models, demonstrating that they maintain a "global workspace" analogous to human cognition. Utilizing a novel interpretability technique called the Jacobian lens (J-lens), the study reveals that these models possess a privileged set of verbalizable internal representations. This small, evolving set of concepts is distinct from their general processing, allowing the models to perform flexible reasoning and reporting. The J-lens effectively surfaces these representations in response to prompts, uncovering the underlying thought processes that guide model behavior, even when such reasoning is not explicitly reflected in their outputs. This finding is noteworthy for the AI/ML community as it may bridge gaps between cognitive science and artificial intelligence, suggesting that LLMs have a form of "access consciousness." The research shows that these verbalizable representations not only enhance model interpretability but also hold potential for improving model safety and alignment. The study proposes a novel training technique called counterfactual reflection training, which aims to shape a model's internal thoughts by influencing what it might articulate in future contexts. This method has shown promise in positively affecting model behavior, suggesting that fostering explicit internal reasoning could be crucial for developing more reliable and ethically aligned AI systems.