Show HN: OSS, K8s-native AI platform for distributed multi-model inference (github.com)

A new open-source AI platform, shaide, has been announced, designed for distributed multi-model inference on self-hosted Kubernetes clusters. This platform aims to simplify the deployment of complex AI infrastructure by providing a comprehensive stack that includes an inference engine, serving orchestrator, gateway, model registry, storage, and observability—all installable with just a single command. Shaide is particularly significant for organizations that require strict data sovereignty, as it operates entirely within their infrastructure, making it ideal for regulated industries and environments that cannot connect to external cloud services. Shaide manages its entire infrastructure as code using Pulumi, allowing for version-controlled and reproducible deployments. It supports multi-model routing and load balancing, facilitating efficient utilization of resources for concurrent tasks. The platform is compatible with existing OpenAI SDKs, which enables seamless integration for users familiar with those tools. By promoting sovereign data practices and simplifying the deployment of AI workloads, shaide addresses a critical need within the AI/ML community to maintain control over sensitive data while leveraging advanced machine learning capabilities.