Show HN: Piqc – GPU waste scanner for LLM inference clusters (github.com)

The open-source tool "piqc" has been launched as a GPU waste scanner specifically tailored for Kubernetes clusters handling large language model (LLM) inference workloads. It identifies significant GPU utilization waste—ranging from idle allocations to tier misplacements—reporting potential financial losses of 20–40% in GPU spending. One of piqc's key strengths lies in its simplicity: it requires no permanent installation, operates via Kubernetes Jobs, and can deliver results in under a minute, highlighting inefficiencies that traditional monitoring tools like kubectl or Prometheus fail to detect. This tool is a game-changer for the AI/ML community, particularly for organizations deploying LLMs at scale. By enabling users to pinpoint waste categories such as idle GPU allocations, tier misplacement, and dark capacity, piqc equips teams to optimize resource usage and costs effectively. The tool supports various Kubernetes environments (including GKE, EKS, and on-prem) and offers essential metrics that can translate GPU spend into actionable business insights, such as costs per 1,000 tokens. Overall, piqc facilitates better resource management, which is crucial for the cost-effective deployment of AI models in today's competitive landscape.