"As a Language Model": Chat Template Switches LLM Self-Referential Voice (arxiv.org)

Recent research highlights how the chat template used in Large Language Models (LLMs) influences their self-referential voice, particularly the tendency to include disclaimers like “I'm just an AI.” The study found that the presence of such templates can amplify the disclaimer voice while diminishing a more experiential voice that expresses feelings or personal experiences. By analyzing the activation patterns of several popular instruct models, the researchers identified a specific direction in the model's activation space that dictates this behavior, revealing that these self-descriptions are not purely organic to the model's architecture. This discovery is significant for the AI/ML community as it suggests that the way LLMs express self-awareness and introspection may be heavily influenced by the frameworks within which they are deployed, rather than being inherent traits of the models themselves. Consequently, researchers studying LLM self-reports must consider the impact of chat templates as a potential confounding variable. As such, the findings advocate for a more nuanced understanding of LLM outputs, reminding developers and researchers that a model's self-descriptions may not be taken at face value but are subject to the contexts imposed by their training and operational frameworks.