🤖 AI Summary
AI is getting very good at clean, well-defined programming tasks—examples include Claude Code and OpenAI’s reported gold at the 2025 International Olympiad in Informatics—but real-world software engineering remains a different beast. The piece argues that current LLMs are still “playing Go” (deterministic, perfect-information, narrow problems) rather than “StarCraft” (real-time, high-dimensional, imperfect-information, multi-agent strategy). Just as AlphaGo solved a constrained board game while AlphaStar required years to master the messy dynamics of StarCraft 2, coding in large, legacy codebases or on distributed cloud systems demands experience, probabilistic reasoning, observability, and coordination that LLMs today lack. A randomized trial even found developers felt more productive with AI but actually took longer on tasks, underscoring limits in complex settings.
Technically, the gap stems from huge state/action spaces, hidden state (“fog of war”), non-determinism, and the infeasibility of fitting entire systems or long logs into model context windows. The article calls for realistic sandboxes, agentic models with tool-calling (to run tests, query logs, and act), and new benchmarks that simulate multi-service workloads, observability data (logs/metrics/traces), and failure modes. Closing this gap is essential not just for coding but for applying AI to any domain where mess, uncertainty, and distributed failure dominate.
Loading comments...
login to comment
loading comments...
no comments yet