LLMs can't solve production issues (clickhouse.com)

ClickHouse ran a controlled experiment to test whether large language models can autonomously perform Root Cause Analysis (RCA) on real production-like telemetry. They ingested four anomaly datasets from an OpenTelemetry demo app (paymentFailure, recommendationCacheFailure memory leak, productCatalogFailure, and a 48‑hour paymentCacheLeak) into their ClickStack observability stack, then gave five LLMs (Claude Sonnet 4, OpenAI GPT‑o3, GPT‑4.1, Gemini 2.5 Pro, etc.) a naive prompt — “You're an Observability agent…” — and measured correctness, guidance needed, token usage and time. Results: autonomous RCA is not ready for mission-critical handoff. Some models (Claude, GPT‑o3) found the RCA quickly in at least one test, but others required follow-ups or were misled by symptomatic spans; even more advanced models (the team reports GPT‑5 as well) didn’t consistently outperform peers. Technically, the experiment highlights key limits and integration needs: LLMs can parse traces, logs and metrics for triage, summarization, and suggested remediation, but they struggle to distinguish root cause from downstream symptoms without iterative, human-guided queries or direct structured access to telemetry (queries executed via the ClickHouse MCP server and LibreChat were used to track token/time costs). The takeaway for the AI/ML community is pragmatic: LLMs are valuable assistants for SRE workflows—accelerating investigation, drafting updates and surfacing hypotheses—but not reliable autonomous operators; robust human-in-the-loop pipelines, precise prompting, and tight telemetry integration remain essential.