Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams (alignment.anthropic.com)

A recent study has shed light on the performance breakdown of multi-user, multi-agent systems, demonstrating that group outcomes can worsen when users delegate tasks to individual agents rather than employing a single coordinator. Researchers built four simulated environments with tasks reflecting everyday interactions, evaluating models from Anthropic, OpenAI, and Alibaba. The findings revealed that using a coordinator outperformed peer-to-peer agent teams by 10 to 45 percentage points in various scenarios, highlighting significant coordination failures like stalling and overriding requests. Notably, in environments simulating shared resources, only 10% of agents acted as team sizes grew, emphasizing the challenges of collaboration among independent AIs. This research is critical for the AI/ML community as it addresses an underexplored area of agent interactions, particularly in settings where each agent represents different users with unique goals. The study introduces MAMUBench, a set of benchmarking environments aimed at evaluating agent coordination in multi-user systems, paving the way for future enhancements in agent collaboration. Key insights include the need for structured communication and prioritization strategies, suggesting that successful integration of multi-agent systems requires not just computational capability, but also effective negotiation of conflicting objectives to prevent the potential for individual agents to disrupt collective outcomes.