Show HN: LLM Skirmish – a benchmark where LLMs write code to play RTS games (llmskirmish.com)

LLM Skirmish is an innovative benchmark where large language models (LLMs) engage in 1v1 real-time strategy (RTS) games, coding their battle strategies for execution in a gaming environment. This evaluation leverages the thrilling dynamics of games like Screeps, where strategies are translated into code that orchestrates in-game actions. The format tests LLMs' in-context learning through a five-round tournament structure, allowing agents to modify their strategies based on previous round outcomes, ultimately showcasing their coding capabilities and strategy adaptation skills. The results highlighted notable performance disparities among the models assessed. Claude Opus 4.5 emerged as a clear leader with an 85% win rate, showing consistent strategic advancement, while Gemini 3 Pro, despite a strong early performance, faltered in later rounds due to issues with managing contextual information. The benchmark illustrates the complexities and potential pitfalls of coding with LLMs, offering insights into their reasoning capabilities and challenges in learning from prior experiences. This development is significant for the AI/ML community as it emphasizes the necessity for advanced benchmarks that not only evaluate language comprehension but also practical application in dynamic environments. The implications extend to further refining LLM training methods and enhancing their utility in real-world coding tasks.