The Complexity of Building an AI SRE (cleric.ai)

Building a reliable AI SRE is far harder than hooking an LLM to observability tools—the article argues that production environments are dynamic, stateful, and full of hidden, idiosyncratic relationships that break naive automation. Real incidents are combinatorial: root causes can be deployments, cron jobs, resource tipping points, or interactions among all three, and data like architecture diagrams or postmortems are often incomplete or ambiguous (one team processed 200+ postmortems and found only ~12 with clear root causes). Practical constraints—rate-limited queries, noisy metrics, and strict access control—mean an AI that chases the first correlation or gives overconfident diagnoses will quickly lose engineers’ trust. The authors share concrete technical approaches from building Cleric: construct an implicit knowledge graph of real service topology (not just documented dependencies), run multiple hypotheses in parallel rather than sequentially, and compute a compound confidence score that favors deterministic signals (topological locality, independent evidence) over simple correlations. They also emphasize best practices—better query patterns (e.g., week-before rollups for normalization), careful RBAC and credential management, and explicit uncertainty reporting. For ML/AI researchers, the takeaway is clear: operational troubleshooting demands causal reasoning, continuous knowledge maintenance, and bespoke training data—challenges that call for new methods beyond conventional supervised LLM fine-tuning.