A secure cloud vault and usage-tracking service for all your LLM providers (blog.mozilla.ai)

The recent launch of any-llm v1.0 introduces a secure SDK and a centralized platform tailored for managing multiple LLM providers via a unified interface. Key to this release is the new zero-knowledge API-key vault and usage-tracking service, which keeps API keys encrypted on the client-side while providing comprehensive analytics without ever exposing user prompts or responses. This architecture ensures maximum privacy and security, making it particularly significant as organizations increasingly seek ways to harness LLM capabilities without compromising sensitive information. The any-llm Managed Platform enhances operational efficiency by enabling real-time budget tracking and cost management across major providers like OpenAI, Anthropic, and Google. Users can create project-based key management and utilize a cryptographic challenge-response system to authenticate and retrieve provider keys securely. This advanced security framework, based on client-side authenticated encryption, positions any-llm as a pivotal player in the AI/ML community, catering to businesses that prioritize compliance with data privacy regulations while leveraging the capabilities of diverse LLM offerings. The service is currently in alpha, inviting early adopters to join a waitlist for testing.