LLM architecture has evolved from GPT-2 to GPT-OSS (modal.com)

OpenAI has introduced gpt-oss, its first open-weight language model since GPT-2, marking a significant leap in LLM architecture. Released under the Apache 2.0 license, gpt-oss is available in two variants, gpt-oss-120b and gpt-oss-20b, both of which boast advanced features like Chain-of-Thought reasoning and low memory requirements, enabling efficient inference with just 16GB of RAM. gpt-oss-120b has been claimed to achieve near-parity with OpenAI's closed-source models, creating a competitive option for developers seeking open-source solutions. The importance of gpt-oss lies in its modernized architecture, which eliminates outdated techniques like dropout and replaces them with more efficient alternatives such as Swish activation functions and RMSNorm normalization. Additionally, the incorporation of Mixture-of-Experts (MoE) and Sliding-Window Attention optimizes memory and compute usage, paving the way for better performance despite having fewer parameters compared to competitors like Alibaba's Qwen3. With impressive benchmarks and a focus on democratizing AI access, gpt-oss empowers developers to innovate more freely, presenting an attractive option for those looking to leverage open-source AI technologies.