Show HN: Pulsys – Pull-through cache for Hugging Face built with io_uring (github.com)

Pulsys has launched an innovative authenticated pull-through cache for the Hugging Face Hub, designed to enhance the efficiency of model downloads. By setting up the cache as a local endpoint, the initial download populates a disk cache that allows subsequent requests to be served directly from disk without incurring additional bandwidth costs. Leveraging advanced technologies like io_uring for Linux systems and the sendfile mechanism for macOS, Pulsys delivers impressive performance metrics, sustaining up to 1.36 million requests per second at 4 KiB and achieving data transfer rates of 90 GB/s at 16 MiB on high-performance hardware. This development is significant for the AI/ML community as it streamlines access to Hugging Face models, mitigating the latency and bandwidth considerations associated with frequent downloads. The seamless integration with existing libraries and tools, such as huggingface_hub and transformers, ensures that users can adopt Pulsys without major modifications to their workflow. Additionally, the project is backed by a structured architecture that includes future plans for multi-node clustering, which could further enhance scalability and reliability for high-demand applications.