RAG is broken, lets fix it (decompressed.io)

Recent discussions in the AI community have highlighted a critical issue known as embedding drift, which occurs when the same text generates different vector representations over time due to factors like model updates and preprocessing changes. This drift can significantly degrade the performance of retrieval-augmented generation (RAG) systems, leading users to perceive a decline in relevance, even when the underlying models remain unchanged. The article emphasizes that unlike typical errors, embedding drift doesn’t trigger alerts, making it a silent threat to retrieval quality. To address this issue, the author suggests best practices for detection and prevention. Key recommendations include consistently checking cosine distances of embeddings to monitor drift, maintaining strict version control for embeddings, and avoiding mixing different embedding generations within a single vector store. By treating the embedding pipeline as a build system, teams can implement a disciplined approach to versioning and re-embedding, ensuring that retrieval quality remains reliable. Ultimately, the article serves as a reminder that vigilance and structured methodologies are essential in maintaining the integrity of AI retrieval systems as they evolve.