Sera-VQ: Discrete codes outperform PCA under extreme embedding compression (github.com)

The introduction of SERA-VQ (Structured Embedding Representation via Residual Approximation and Vector Quantization) marks a significant advancement in the field of AI and machine learning by demonstrating that discrete codes can outperform traditional dense embeddings under extreme memory constraints. While modern systems often rely on dense floating-point vectors for representing semantic similarity, SERA-VQ reveals that in low-memory situations (≤32 bytes), discrete representations provide a substantial improvement in retrieval tasks. For instance, SERA-VQ, which compresses embeddings into sequences of discrete codes using PCA and Residual Vector Quantization, achieved a 24% relative improvement in ranking quality compared to PCA-compressed embeddings. This breakthrough is notable because it challenges the prevailing assumption that dense embeddings are universally superior. The technique not only enables extreme compression—from 1536 bytes down to just 8–32 bytes—but also preserves semantic structure, enhancing performance in real-world retrieval evaluations like BEIR (SciFact) and STS-B. As researchers prepare to publish their findings on arXiv, SERA-VQ presents an exciting alternative for situations where memory is at a premium, pushing the boundaries of how we think about representation in machine learning.