Asymmetry of Verification and Verifier's Rule (www.jasonwei.net)

Asymmetry of verification is the observation that many tasks are far easier to check than to solve — think Sudoku or code judged by unit tests — and that this gap is becoming central as reinforcement learning scales. The author formalizes this into a “verifier’s rule”: the easier a task is to verify, the easier it is to train AI to solve it. Concretely, tasks that are most tractable for AI tend to have (1) objective truth, (2) fast verification, (3) scalability of verification, (4) low verification noise, and (5) a continuous reward signal (or an aggregate that approximates one). Practical techniques (answer keys, test suites) can widen the verification gap, which is why many popular benchmarks have been cracked: they’re designed to be verifiable at scale. The piece highlights AlphaEvolve (Google) as a real-world instantiation of ruthless guess-and-check that exploits verification asymmetry to optimize single problems (train = test), producing mathematical and operational wins. The author stresses this concept is distinct from P vs NP: verifier’s rule doesn’t claim polynomial-time solutions, only that if you can cheaply measure quality you can optimize via RL and search. Implication: AI progress will cluster on highly verifiable tasks, producing a “jagged edge” of intelligence unless we engineer quick, scalable measurements for harder physical or scientific objectives.