Why Large Language Models Won't Replace Engineers Anytime Soon (fastcode.io)

Recent arguments synthesize why Large Language Models (LLMs) — despite dazzling capabilities in code, papers, and design — are unlikely to outright replace engineers. LLMs are trained as next-token predictors under maximum likelihood, so they optimize plausibility of text rather than causal correctness. Engineering, by contrast, is best modeled as reinforcement learning (RL): agents take actions, observe delayed outcomes, and optimize long-horizon reward J(θ)=E[∑γ^t r_t]. Techniques like RLHF add human preference rewards but still reward “sounding right” not actually working. Core mathematical limits remain: gradient descent finds local optima, temporal credit assignment becomes noisier as horizon H grows (Var[∇J] ∝ H), and the RL “deadly triad” (function approximation, bootstrapping, off-policy data) makes stable learning hard. From a formal view, an LLM trained on text lacks estimators for environment dynamics P(s_{t+1}|s_t,a_t) in POMDP tasks, so it cannot compute truly optimal actions. For the AI/ML community this reframes priorities: build systems that interact with environments, improve long-horizon credit assignment, and focus verification and causal modeling rather than pure scale. Practical barriers — subjective reward modeling, massive simulation cost, unverifiable neural outputs — plus real-world failures (Tesla’s over-automation, Duolingo’s AI-first push) show humans still outperform isolated automation. In short, LLMs are powerful amplifiers of engineering work, not substitutes for the causal reasoning, experimentation, and responsibility that engineers provide.