GGUF: GGML Universal File (en.wikipedia.org)

The GGUF (GGML Universal File) format, developed by Georgi Gerganov and introduced on August 22, 2023, significantly enhances how machine learning tensors and metadata are stored, allowing for more efficient model data management. This binary file format consolidates both tensors and the necessary metadata into a single file, which improves performance by facilitating faster saving and loading of model data. By introducing a structured key–value metadata system, GGUF replaces its predecessor formats (GGML, GGMF, GGJT) and maintains compatibility with diverse model architectures without compromising backward compatibility. GGUF has quickly become the standard for distributing quantized large language models for local inference, supported by tools like llama.cpp, Ollama, LM Studio, and Hugging Face, which now hosts tens of thousands of GGUF checkpoints. The format's design allows for self-contained files that can be memory-mapped, enhancing access to large models beyond available memory limitations. Moreover, the inclusion of various quantized integer types and floating-point formats, alongside the extensibility of the metadata system, positions GGUF as a versatile solution in the evolving AI/ML landscape, streamlining model deployment and usage across multiple platforms.