We Politely Insist: Your LLM Must Learn the Persian Art of Taarof (arxiv.org)

Researchers introduce TaarofBench, the first benchmark to teach and evaluate large language models (LLMs) on Persian taarof — a nuanced system of ritual politeness emphasizing deference, modesty and indirectness. TaarofBench contains 450 role-play scenarios across 12 common social topics, validated by native speakers, and was used to test five frontier LLMs. The study finds large cultural competence gaps: when taarof is appropriate, model accuracy lags native speakers by 40–48%. A human baseline study with 33 participants (11 native Persian, 11 heritage, 11 non‑Iranian) anchors these gaps and highlights performance variation across topics, gender-based asymmetries, and consistent gains when prompts are in Persian. Key technical takeaways: conventional “politeness” metrics often miss taarof-specific norms, so Western politeness frameworks can produce outputs that are technically polite but culturally inappropriate. Supervised fine-tuning raised alignment with taarof expectations by 21.8%, while Direct Preference Optimization (DPO) produced a 42.3% improvement, showing targeted training and preference learning can substantially close the gap. The work underscores the need for culturally specific benchmarks, multilingual/contextual prompting, and tailored alignment methods to build LLMs that navigate complex social norms safely and effectively in global deployments.