Cost-Effective, Low Latency Vector Search with Azure Cosmos DB [pdf] (www.vldb.org)

Microsoft researchers integrated the DiskANN vector indexing library directly into Azure Cosmos DB NoSQL, delivering an operational, cloud-native vector search that avoids separate vector databases. The integrated system stores vector index terms inside Cosmos DB’s Bw-Tree per partition (one vector index per replica), supports immediate durable updates, and leverages Cosmos DB’s multi-tenancy, automatic partitioning and pay-per-use/serverless cost model. For common production scales (10M 768-D vectors) the authors report ~20 ms end-to-end query latency (including fetching documents), stable recall across long update sequences, and query cost roughly 43× and 12× lower than Pinecone and Zilliz serverless tiers respectively — while scaling out to billions of vectors. Technically, DiskANN provides a graph-based, quantized index optimized for SSD-first operation: most accesses touch compact quantized representations and full-precision vectors are read ~50× less often, enabling low-memory replicas and efficient SSD-backed search. Bw-Tree stores index terms so hot partitions are cached and cold collections avoid memory pressure; updates do not require costly index rebuilds or segment merging. The design yields logarithmic query complexity, query cost that grows <2× even when index size increases 100×, supports hybrid predicate+vector queries, and eliminates cross-system replication/divergence. The result is a cost-efficient, low-latency vector search option tightly coupled to transactional data — simplifying operations and improving resilience for semantic search, recommendations, and AI-agent workloads.