J-Space: Yet Another LLM Mind Reader? (huggingface.co)

A new research paper introduces "J-Space," or the Jacobian lens (J-lens), unveiling a novel method for exploring the inner workings of large language models (LLMs) like Anthropic's Claude 3.5. This study distinguishes itself from previous attempts by systematically demonstrating that LLMs possess separate representational routes for language processing and explicit reporting, indicating a sophisticated organization of internal representations. The researchers conducted experiments where they exchanged internal representations of languages, finding that Claude maintained effective language processing even when prompted with a different language, showcasing at least two distinct cognitive pathways. The J-lens technique significantly enhances mechanistic interpretability by providing a linear approximation of a model's computations, allowing researchers to identify and manipulate how concepts are represented and used during reasoning tasks. This method offers causal evidence that certain representations are actively engaged in model outputs, rather than purely correlational. For example, interventions in Claude's activations produced observable changes in its outputs, further bridging the gap between concepts and their computational roles. The findings spark interest in the AI/ML community as they pave the way for more sophisticated understandings of model behavior and representation, while inviting deeper inquiry into the nature of LLM cognition and interpretability.