Evaluating Theory of Mind and Internal Beliefs in LLM-Based Multi-Agent Systems (arxiv.org)

Recent research has explored the integration of Theory of Mind (ToM) and Belief-Desire-Intention (BDI) models into large language model (LLM)-based multi-agent systems (MAS), aiming to enhance collaborative problem-solving. Despite their promise, achieving effective collaboration in dynamic environments remains challenging due to variable performance among LLMs. This study investigates how internal beliefs and cognitive mechanisms, alongside formal logic verification, affect decision-making and coordination in LLM-based systems. The researchers have developed a novel multi-agent architecture that incorporates ToM, BDI-style internal beliefs, and symbolic solvers, which they tested in a resource allocation scenario using different LLMs. Their findings highlight complex interactions between the cognitive mechanisms and the performance of the LLMs, emphasizing that simply adding cognitive features does not guarantee improved outcomes. This work contributes significantly to the AI community by proposing a comprehensive framework for augmenting collaborative intelligence in multi-agent systems, addressing key gaps in understanding how various components interact to enhance system accuracy and effectiveness.