Show HN: StarSkirmish, an arena where LLMs create StarCraft Brood War bots (starskirmish.com)

🤖 AI Summary
StarSkirmish has launched as an innovative benchmark that challenges large language models (LLMs) to program bots for the classic game StarCraft: Brood War. Each model is allocated one hour to create a Protoss bot using C++, with the finished bots competing against one another and established human-written counterparts. This setup evaluates the LLMs on their long-horizon reasoning and agentic coding skills, as bots must learn and adapt based on practice matches. The results offer a clear score in terms of win rates, allowing easy comparison against top human-written bots. This new benchmark is particularly significant for the AI/ML community as it provides a rigorous framework to assess complex coding abilities and strategic thinking inherent in programming tasks. The top-scoring models, including GPT-6 Astra and Claude Opus 5.5, exhibit exceptional performance, suggesting advancements in AI-driven coding while also correlating positively with existing programming benchmarks like Vals IOI and WeirdML. The StarSkirmish emphasizes the importance of long reasoning periods for optimal performance, hinting at future iterations that will explore even more extended runtimes to further test and enhance AI reasoning capabilities.
Loading comments...
loading comments...