Mindstorms in Natural Language-Based Societies of Mind (NLSOM) (github.com)

The Natural Language-Based Societies of Mind (NLSOM) concept has been introduced and accepted by CVMJ 2025, gaining recognition as the Best Paper at the NeurIPS 2023 Ro-FoMo Workshop. NLSOM seeks to enhance collaborative problem-solving by allowing multiple agents—comprised of large language models (LLMs), neural network-based experts, APIs, and role-players—to communicate in natural language. This framework utilizes a "Mindstorm" approach, where agents interview one another, significantly improving multimodal zero-shot reasoning and the overall efficiency of task resolution compared to previous models that depend on a singular entity. The significance of NLSOM lies in its modular design, enabling easy integration of new agents into the system as users define their tasks. By implementing a reward mechanism to evaluate agent contributions, NLSOM not only fosters competition among models but also establishes a merit-based environment. This innovation positions NLSOM as a promising solution for non-technical users, providing tailored recommendations for communities and agents that can collectively address user requests without requiring extensive technical knowledge. As a foundational step, the NLSOM project aims to streamline complex problem-solving processes within the AI/ML community, marking a significant evolution in collaborative AI methodologies.