Which flavor of software are LLMs exactly? (silvestreperret.com)

The recent discussion around Large Language Models (LLMs) highlights their unique role in the evolving landscape of software development, diverging from traditional Machine Learning paradigms. While LLMs are built on advanced Neural Network architectures like Transformers, which brought significant training efficiency and scalability since their introduction, they also challenge the established notion of "Software 2.0." This concept, introduced by Andrej Karpathy, suggests that instead of writing explicit instructions for computers, data-driven learning models automatically generate behavior through tuned weights. However, LLMs introduce a new layer—allowing users to steer their behavior through natural language inputs, prompting a potential reclassification into what Karpathy terms "Software 3.0." The implications of LLMs stretch far beyond mere technical advancements. As they enable non-experts to harness powerful AI capabilities through prompt and context engineering, the skillsets of software engineers are shifting from traditional coding to optimizing interactions with these models. This paradigm shift raises questions about the future of professions in AI, software development, and communication, while also presenting challenges like mental health risks and environmental impacts. Overall, as LLMs redefine how AI is utilized, understanding their operations and implications becomes crucial for adapting to this transformative era.