Surfacing LLM Biases Through Graffiti (nullpxl.com)

While visiting Niterói, Brazil, the author tried to decode faded street markings and used multimodal LLMs for help. The tags read “ROTA DARWIN” (a trail-marker), but GPT-5 confidently claimed the graffiti said “ACAB” and Claude / Sonnet 4.5 offered a different politically charged reading (“EAT THE RICH”). The mismatch highlights how LLMs can impose high-probability, politically loaded interpretations onto ambiguous visual input—likely because training data disproportionately contains politically themed graffiti and the models default to those priors when uncertain. The author notes three recurring failure modes: models favor “likely” answers over atypical but correct ones, they poorly express calibrated uncertainty, and in some cases providing less context can reduce misleading inferences. For the AI/ML community this is a compact case study in multimodal hallucination and bias: ambiguous visual cues + skewed training distributions lead to confident, incorrect semantic labels. Technical implications include the need for better confidence calibration, abstention mechanisms, and balanced multimodal datasets; designers should also evaluate models on out-of-distribution, low-visibility imagery and incorporate human-in-the-loop verification for real-world perception tasks. Practically, prompt engineering, uncertainty-aware outputs, and provenance metadata can reduce harm when systems interpret noisy or culturally specific visuals.