I built a fleet-scale inference control plane using Crossplane (blog.crossplane.io)

A new project called Modelplane has been launched, utilizing Crossplane to enable the management of GPU fleets for model inference across multiple cloud environments and on-premise setups. This innovative control plane allows platform teams to create a unified infrastructure where machine learning (ML) teams can seamlessly deploy models, ensuring stable endpoints that are compatible with OpenAI's standards. The significance of Modelplane lies in its role in democratizing access to advanced inference capabilities, as open-weight models shift from high-end labs to a broader audience, including regulated enterprises and cost-sensitive businesses. At its core, Modelplane operates entirely on Crossplane's framework without bespoke controllers, leveraging its powerful composition capabilities for scaling across diverse infrastructures. By using a declarative approach, it streamlines processes such as cluster provisioning and model deployment while addressing the complexities of hardware allocation and cost optimization. The project exemplifies advancements in developer experience with new Crossplane tools, including a CLI that simplifies function writing in Python—connecting ML practitioners more closely with cloud-native technologies. Through Modelplane, the AI/ML community can anticipate more accessible and efficient deployment of machine learning models in a wide variety of environments, bridging significant gaps left by traditional inference systems.