The Karpathy Principle (olshansky.info)

“The Karpathy Principle” names a common engineering truth in AI: getting a model to look good in demos is easy compared with making it reliably useful in production — the final sliver of correctness or reliability (the “last 0.001%”) consumes the vast majority of time and effort. Coined after a widely cited Andrej Karpathy / Dwarkesh interview, the phrase reframes the “demo-to-product gap” and “march of nines” into a single observation about diminishing returns: incremental gains near perfection require disproportionate investment. For practitioners this matters because research metrics (average accuracy, a leaderboard score) often understate what production systems demand: robustness to distribution shift, calibration, adversarial and long‑tail cases, latency and throughput guarantees, monitoring, continual retraining, and human-in-the-loop feedback (e.g., RLHF). The principle implies that deployment work — data curation, instrumentation, safe-fail behaviors, automated validation, and handling corner cases — dominates costs and shapes design choices long before models reach business or safety targets. Recognizing this helps teams prioritize engineering, evaluation protocols, and risk management rather than chasing small benchmark improvements that don’t translate to real-world reliability.