Swapping pgvector for turbopuffer made our whole database faster and cheaper

Modem, a company leveraging AI embeddings for customer message analysis, faced significant slowdowns as their user base grew. Initially using the pgvector extension in Postgres, query performance deteriorated, particularly for smaller customers whose data was pushed out of memory, leading to delays of up to 28 seconds. Attempts to optimize with HNSW indexing resulted in worse performance due to its inability to combine filters effectively in a multi-tenant environment. Ultimately, they transitioned to a dedicated vector database solution, turbopuffer, which improved search times significantly by efficiently handling filters and maintaining cost-effective storage for embeddings.

The switch to turbopuffer, a proprietary SaaS created by former Shopify engineers, allowed Modem to keep vectors organized by customer, ensuring faster access and better cost management. This approach addresses the limitations of pgvector by performing searches within specific filters upfront, rather than discarding irrelevant results afterward. As a result, Modem was able to enhance their database performance, making searches faster and more efficient for all customers while simultaneously reducing their operational costs.