AI Has Agents but No Agency (niketpatel.com)

Researchers and commentators are arguing that today’s large language models (LLMs) — exemplified by GPT-4 — are extremely powerful memory-and-pattern engines but lack true agency: the capacity to think independently or invent concepts that lie outside their training distribution. Technically, LLMs excel at recognizing statistical regularities and recombining vast amounts of stored text (interpolation and sophisticated retrieval), which produces fluent, contextually apt outputs. That capability enables summarization, translation and creative-seeming generation, but it’s fundamentally derivative: models struggle to produce genuinely novel theoretical frameworks or solve problems that require conceptual leaps beyond seen data. This gap matters because the field has leaned heavily on scale (more parameters, more data, more compute) — at enormous cost (training GPT-4 reportedly exceeded $100M) — and is hitting diminishing returns. Proposed fixes like continuous learning mostly keep models contemporaneous with human output rather than giving them independent creativity. The implication for AI/ML is that progress will likely require new architectures or learning paradigms that let memory support, rather than constrain, reasoning — enabling emergent, non-derivative problem-solving instead of ever-larger repositories of recycled knowledge.