The Not-So Bitter Lesson (blog.mariusvach.com)

Richard Sutton’s “Bitter Lesson” — that general methods exploiting search and raw compute beat hand-crafted, domain-specific solutions — is reframed here as a practical engineering blueprint rather than a doom prophecy. The author’s turning point came from George Hotz’s tinygrad: the library demonstrates that making code fast across hardware is itself a multi-scale search problem (scheduling, kernels, etc.) that you must expose to exploit compute. Once you accept “search” as learning algorithms exploring solution spaces (not web search), the value proposition for engineers becomes clear: design the problem, then let compute do the heavy lifting. Practically, that means three core engineering tasks: (1) formulate problems to expose the right parameters, (2) build robust evaluations to guide search, and (3) define constraints (latency, memory, safety). Examples range from tinygrad and DSPy (abstractions/signatures that separate problem spec from solver), to SAT solvers requiring CNF formulations, to AI evals for LLM apps, and even AlphaFold or fusion design where simulations define constraints and compute explores designs. The implication for AI/ML teams is a shift toward building scalable harnesses—representations, eval pipelines, and constraints—that make search tractable and safe, not writing bespoke heuristics. Engineers aren’t obsolete; they’re the architects who expose the search problems that compute will solve.