Tricks from OpenAI GPT-OSS you can use with transformers (huggingface.co)

OpenAI’s recent release of the GPT-OSS series introduces a suite of advanced techniques now integrated into the Hugging Face transformers library, significantly enhancing efficiency and accessibility for large language models. Key innovations include MXFP4 quantization, which compresses model weights into a novel 4-bit floating format with blockwise scaling, drastically reducing memory use to enable running massive models like GPT-OSS 120B on a single GPU. This is paired with precompiled, system-tailored custom kernels hosted on the Hugging Face Hub that optimize critical operations such as RMSNorm, Mixture of Experts layers, and Flash Attention 3 with attention sinks, offering substantial speedups and improved throughput across common transformer architectures. These upgrades solve long-standing challenges around dependency bloat and compilation complexity by dynamically downloading compatible optimized kernels at runtime, ensuring smooth deployment and consistent performance gains. Transformers now also support advanced parallelism schemes like tensor parallelism (TP) out of the box, enabling seamless sharding of layers across GPUs with an automatic plan to maximize memory efficiency and throughput—even for very large models. Together, these innovations deliver a plug-and-play toolkit that allows researchers and practitioners to fine-tune, run, and deploy state-of-the-art LLMs more effectively, fostering broader experimentation and faster adoption of cutting-edge model compression and parallelization strategies within the AI community.