Debian Inference Portal (inference.debian.net)

The newly launched Debian Inference Portal offers a self-service platform for Debian contributors to access shared large language model (LLM) inference. By logging in with their Salsa accounts, users can create API keys, manage spending, and utilize an OpenAI-compatible proxy without the need for a separate Scaleway account. This initiative is significant for the AI/ML community as it enables contributors to efficiently use generative AI resources while promoting fair usage and accessibility thanks to Scaleway’s sponsorship of a monthly pool of credits. The portal currently supports two models for inference: Zhipu AI's GLM-5.2 and DeepSeek's V4 Flash, with different pricing structures and capabilities. The shared service incorporates a weekly budget system to manage usage, ensuring that resources are available to all contributors. Notably, while DeepSeek's model offers a more cost-effective option, GLM-5.2 charges a full rate for cached tokens, which could escalate costs in extensive scenarios such as software development workflows. This innovative solution not only fosters collaboration within the Debian community but also underscores the growing importance of shared resources in the rapidly evolving landscape of AI and machine learning.