Building the AI Retrieval Infrastructure Behind 20B+ Vectors at HubSpot (product.hubspot.com)

HubSpot has announced the successful scaling of its Vector-as-a-Service (VaaS) platform, which now manages over 20 billion vectors across more than 200 indexes. Initially launched as a proof of concept, the VaaS infrastructure has evolved to support critical functions such as semantic search and Retrieval Augmented Generation (RAG) for 38 teams. At its core, HubSpot utilizes the open-source Qdrant vector database, which employs Hierarchical Navigable Small World (HNSW) algorithms for low-latency operations. This architecture is designed for high efficiency, enabling the platform to handle over 5,000 writes per second, with spikes reaching 100,000 RPS. The significance of this development lies in the successful transition from a simple setup to a robust, scalable infrastructure using Kubernetes Operators. This allows HubSpot to automate the lifecycle management of its clusters, significantly reducing operational complexity while enhancing performance and reliability. Key innovations include a namespace-per-cluster strategy for improved isolation and a streamlined process that simplifies cluster scaling and maintenance, mitigating the risks of data loss during operations. The evolution of HubSpot's VaaS platform exemplifies the increasing importance of efficient, robust infrastructure in powering advanced AI use cases in the tech industry.