LLM Council: query multiple LLMs, and asks them to rank each other's work (github.com)

LLM Council is an open‑source, local web app that routes a single user query to multiple LLMs (configurable via OpenRouter), collects their answers, then asks each model to anonymously review and rank the others before a designated “Chairman” LLM synthesizes a final reply. The flow is: (1) first opinions—parallel responses shown in a tab view; (2) review—each model rates anonymized peers on accuracy/insight; (3) final response—the chairman compiles the consensus answer. The repo demonstrates the concept with example models like OpenAI GPT‑5.1, Google Gemini 3 Pro, Anthropic Claude Sonnet 4.5, and xAI Grok 4. For AI/ML practitioners this is a compact experiment in ensemble reasoning, meta‑evaluation, and automated model arbitration: it makes comparing outputs, surfacing disagreements, and producing a consensus answer easy to inspect. Technically it’s a FastAPI + async httpx backend, React + Vite frontend (react‑markdown), JSON file storage, and uses OpenRouter API keys (OPENROUTER_API_KEY) to call models; project management uses uv. The repo is a Saturday‑hack prototype (no support promised), so it’s useful as an inspiration or testbed for multi‑model evaluation, automated model selection, and studying cross‑model critique — but expect tradeoffs in cost, latency, and potential cascading errors if models misrank each other.