I spent my winter break teaching an LLM to play Diplomacy with RL (www.benglickenhaus.com)

A researcher has successfully trained a large language model (LLM), specifically Qwen3-14B with LoRA, to play no-press Diplomacy using reinforcement learning (RL) techniques, achieving an impressive win rate of 80% against a baseline bot, DumbBot. This project, conducted over winter break, utilized Modal’s serverless infrastructure, highlighting the importance of scalable rollout systems and innovative methodologies in RL research. The model's performance surpassed existing benchmarks, such as DipNet, and demonstrated significant improvements in efficiency from a custom logits processor, which enhanced valid move accuracy to reduce the likelihood of generating invalid actions. This work is significant for the AI/ML community as it emphasizes the complexities of training models in adversarial multi-agent environments, where human-like strategic decision-making is crucial. The study explores key challenges in RL, particularly constrained generation and reward assignment methods, which contributed to model improvement. Moreover, the findings open avenues for future research into full-press Diplomacy scenarios that involve natural language negotiations, potentially offering insights relevant to real-world applications in business and international relations. By focusing on a minimalist approach—eschewing traditional search techniques for simpler LLM + RL frameworks—the project provides valuable lessons in building effective AI systems for complex strategic gameplay.