Which Nested Data Format Do LLMs Understand Best? JSON vs. YAML vs. XML vs. Md (www.improvingagents.com)

🤖 AI Summary
Ainews247 tested how well three LLMs (GPT‑5 Nano, Llama 3.2 3B Instruct, Gemini 2.5 Flash Lite) can read deeply nested data presented as JSON, YAML, XML or Markdown. Using synthetic Terraform‑style configs with 6–7 levels of nesting, the team ran 1,000 retrieval questions per format (calibrated so baseline accuracy sat ~40–60%) and measured accuracy, token usage and data size. Key findings: YAML produced the highest accuracy for GPT‑5 Nano (62.1%) and Gemini (51.9%), while XML was consistently the worst (GPT‑5 Nano 44.4%, Gemini 33.8%). Llama 3.2 showed little format sensitivity. Format choice could swing results dramatically — the study reported cases with up to a 54% relative difference in correct answers between formats. Beyond accuracy, token efficiency differed sharply: Markdown (implemented with heading levels for depth) was the most token‑efficient, using roughly 34–38% fewer tokens than JSON and ~10% fewer than YAML; XML required about 80% more tokens than Markdown, greatly increasing inference cost. Practical takeaway: if accuracy is critical and you don’t know your model’s preferences, default to YAML; if cost or latency matters, Markdown is a strong choice; avoid XML for large nested payloads. The authors recommend per‑model testing (results vary by architecture and training data) and note limitations like single‑domain data and relatively simple question types.
Loading comments...
loading comments...