Show HN: FusionHarness – An Open-source Mixture-of-Agents compound-model server (github.com)

FusionHarness has been introduced as an open-source Mixture-of-Agents compound-model server that offers a self-hostable alternative to OpenRouter's Fusion API. This innovative framework allows users to fan out prompts to multiple large language models (LLMs) simultaneously, with a judging mechanism that analyzes their responses for consensus and unique insights. The synthesized final answer, produced from this analysis, demonstrates improved accuracy, often outperforming individual models while being cost-effective by utilizing budget models that can match the performance of more advanced models. This development is significant for the AI/ML community as it enhances the collaborative potential of LLMs, yielding superior outcomes through a structured aggregation of insights. Key technical features include the ability to integrate seamlessly with existing OpenAI clients, flexible configuration options for panel selection, and a built-in evaluation harness for comparing panel performance against solo models. With features such as tool-enabled LLMs and a secure coding environment, FusionHarness provides robust capabilities for researchers and developers to explore and implement advanced AI solutions efficiently.