LLMs could be, but shouldn't be compilers (alperenkeles.com)

A recent discussion in the AI/ML community explores whether large language models (LLMs) could function like compilers, enabling users to write programs in natural language without directly referencing the underlying code. This notion leans on the idea that programming could evolve to a point where human language alone suffices to communicate software specifications, as posited by figures like Andrej Karpathy. However, the author cautions that while LLMs may improve in generating plausible code, their current tendency to "hallucinate" flaws undermines their reliability as a serious abstraction layer for programming, as precise control and guarantees inherent to traditional programming languages are lost. The implications of this shift in programming paradigms are significant. As LLMs become integral to software development, they promise to simplify coding by turning vague specifications into executable code, yet this convenience poses risks. Developers may become less diligent in refining their specifications, allowing the model to make critical design choices for them. This could lead to software that diverges from original intents without the creator’s awareness, fundamentally altering the developer's role from meticulous builder to consumer of generated outputs. The author emphasizes the need for robust specification and verification as essential skills in a landscape where LLMs serve as a "compiler-like" tool, highlighting the potential dangers of ceding too much control to these advanced models.