Fabrice Bellard's TS Zip (2024) (www.bellard.org)

The newly announced ts_zip utility leverages Large Language Models (LLMs) for text compression, achieving impressive compression ratios that surpass conventional tools. By utilizing the RWKV 169M v4 language model, ts_zip compresses text files significantly better than traditional methods, minimizing file sizes while maintaining deterministic output across different hardware and software configurations. However, there are technical caveats: it requires a GPU for optimal speed, and its current performance is considerably slower than conventional compression tools, maxing out at approximately 1 MB/s on high-end GPUs like the RTX 4090. This development is significant for the AI/ML community as it experimentally demonstrates the potential of LLMs beyond natural language processing tasks, venturing into efficient data storage techniques. With specific support for English text and other languages, as well as source code, ts_zip could enhance workflows in data-heavy environments. It also highlights the growing trend of integrating AI models into diverse applications, paving the way for innovative uses of language models in areas traditionally dominated by specialized algorithms.