Lossless model compression experiment: GLM-5.2 in 25% less memory (brianbell-x.github.io)

A recent experiment with the GLM-5.2 model has achieved a noteworthy 25% reduction in memory usage through innovative lossless model compression techniques. Utilizing a K15 charged-format accounting coupled with a byte-split representation, researchers successfully decoded all 59,509 BF16 tensors with matching precision, demonstrating that the new representation costs only 12.005 bits per weight—24.967% less than traditional BF16. This method capitalizes on the repetitive nature of exponent values in trained models, implementing a compact 4-bit code system for the sign and exponent while preserving the raw 7-bit mantissa. The significance of this development lies in its potential to enhance efficiency in AI and machine learning applications, particularly in resource-constrained environments. Moreover, the K15 layout allows for high-performance processing with a notable speedup in GEMV operations, although it doesn’t yet include all optimizations for seamless integration. By positioning this experiment alongside existing methods like ZipNN and DFloat11, it paves the way for further exploration into lossless compression strategies, potentially reshaping how large language models manage memory. This represents a step forward not only in model efficiency but also in maintaining fidelity during data reconstruction.