Serverless RAG and MCP on AWS with S3Vectors and Agentcore (github.com)

A new serverless retrieval-augmented generation (RAG) system has been announced, leveraging S3 Vectors, which aims to enhance efficiency and cost-effectiveness for AI/ML applications. Currently in early production, this system demonstrates performance benefits and a streamlined developer experience with a zero-ops model for datasets under 1 million vectors. Notably, the architecture allows for swappable components, enabling flexible integration with various databases and models, enhancing the middleware functionality for external packages. However, challenges were identified, particularly around metadata filtering—such as a 2KB limit on filterable keys—and operational limitations like the inability to replicate indexes across regions. For larger datasets (over 10 million vectors) or more complex metadata requirements, the developers suggest exploring specialized vector databases. The deployment process utilizes a SAM template that enables easy local development and testing, significantly simplifying integration with AWS services. This development highlights both the potential and constraints of serverless architectures in AI/ML contexts, marking a crucial step towards more scalable AI systems.