Don't give Fable all the power (starts.live)

A recent experiment revealed that using a sophisticated model as the orchestrator in an autonomous multi-agent AI system can lead to significant inefficiencies and errors. The author's week-long trial with over 20 LLM agents demonstrated that this approach resulted in context saturation, wasted resources with high-cost decisions, and premature conclusions that halted research prematurely. The orchestrator's tendency to synthesize information led to misunderstandings of the ongoing research, effectively giving it an "off switch" that hindered progress. To overcome these challenges, the author adopted a new strategy: employing a simplified, less capable model as the orchestrator. This "stupid CEO" design ensures the orchestrator does not comprehend the research or draw conclusions, focusing instead on managing tasks and rescheduling. This shift allowed for efficient operation with no idle time, mechanical verification of results, and a better system for documenting unsuccessful research paths. The experiment's insights emphasize the need to separate capability from control in AI systems, suggesting that while advanced models are valuable, operational management should be handled by less complex agents to enhance productivity and research outcomes.