Silicon Valley bets big on ‘environments’ to train AI agents (techcrunch.com)

Silicon Valley is pouring resources into building interactive reinforcement learning (RL) “environments” — simulated workspaces that let AI agents practice multi-step software tasks (e.g., buying something in a browser) and receive reward signals for success. Big labs are building in-house environments and buying from startups and data-labeling firms, spawning companies like Mechanize Work, Prime Intellect, Mercor and expanded units at Surge and Scale AI. Investors and researchers compare this moment to the dataset rush that powered large language models; Anthropic reportedly considered a >$1B investment, and founders hope to become the “Scale AI for environments.” Technically, environments differ from static datasets by modeling unexpected interactions, tool use, internet access and complex stateful tasks — making them far more expensive and delicate to build and evaluate. They’ve already contributed to recent RL-driven advances (OpenAI’s o1, Claude Opus 4), but scaling remains uncertain: environments demand huge compute, robust evaluation to avoid reward hacking, and careful design to capture edge-case failures. Some groups push open-source hubs and compute marketplaces to democratize access, while skeptics (including Karpathy and former Meta researchers) warn RL’s limits and the practical difficulty of producing universally useful, scalable environments. The outcome will shape whether agentic AI moves beyond brittle demos to reliable, general-purpose assistants.