Show HN: Chatpack – Compress chat exports 13x for LLM analysis (Rust) (github.com)

A new tool called Chatpack has been launched, allowing users to compress chat exports for use in Large Language Model (LLM) analysis. Utilizing CSV format, Chatpack achieves an impressive 13x reduction in size, transforming raw exports—which often contain significant overhead due to metadata—into optimized data sets suitable for model ingestion. The tool supports multiple platforms (Windows, macOS, Linux) and popular messaging formats, such as Telegram, WhatsApp, Instagram, and Discord. This development is particularly significant for the AI/ML community because it addresses the challenge of excessive metadata in chat data, which can waste costly tokens during model training and inference. By effectively reducing the amount of data processed without losing essential information, Chatpack enables more efficient LLM interactions and can enhance the performance of retrieval-augmented generation (RAG) pipelines. Key features include smart merging of consecutive messages, filtering by date or sender, and the ability to output in various formats, making it a versatile tool for researchers and developers working with conversational data.