The Gibraltar Fallacy: How LLM Dashboards Distort Reality (oblsk.com)

“The Gibraltar Fallacy” warns that clean LLM evaluation dashboards can be as misleading as a 50-year naval record that misplaced a sunk U-boat. The piece uses the U-869 story to illustrate how off-the-shelf scoring systems produce tidy aggregates that seem to confirm model quality, while the model’s real-world behavior — the “U-Who” running in production — can be entirely different. This is framed as the “90% accuracy” trap: high aggregate scores can coexist with catastrophic single-output failures that define user experience. Technically, the essay critiques common automated scorers that measure surface-level keyword overlap with references rather than semantic correctness, context awareness, or safety. Such metrics are proxies, not ground truth; optimizing to them creates systemic blind spots (hallucinations, PII leaks, invented legal clauses) that dashboards won’t catch. The takeaway for practitioners is practical: dashboards and aggregate numbers are useful but insufficient — teams must complement them with case-level, manual error analysis (teased as “The Shadow Divers Method”) to surface real failure modes before deployment.