Nvidia AIStore – scalable storage for AI applications (aistore.nvidia.com)

Nvidia has announced AIStore (AIS), a scalable and versatile distributed storage solution designed specifically for AI applications. This innovative storage stack can be deployed in various settings, from a single Linux machine to extensive bare-metal clusters, and it supports flexible scaling and consistent performance without relying on caching mechanisms for remote data. AIS’s features, such as multi-cloud access, Kubernetes compatibility, and a robust API, enhance its functionality for developers and data scientists working with complex datasets. The significance of AIS for the AI/ML community lies in its ability to handle large-scale data workloads efficiently, offering reliable I/O distribution and fast data access. Key capabilities include a unified namespace for seamless dataset management across clusters, high availability through self-healing mechanisms, and an advanced chunked object storage format tailored for machine learning workflows. With support for batch operations and ETL (Extract, Transform, Load) processes, AIS aims to streamline data access and transformation tasks, ultimately accelerating the development of AI models by simplifying the data handling process.