ASCII art benchmark to compare how well LLMs create art using text characters (www.asciibench.com)

ASCII Bench is a lightweight new benchmark that pits large language models against a concrete creative task: produce ASCII art from a short text prompt. The example shown — “Create ASCII art of a skull wearing a crown” — yields two very different model outputs, one blocky and geometric, the other ornate with decorative framing. By framing ASCII generation as a measurable task, the benchmark makes it easy to compare fidelity to the prompt, structural coherence, use of whitespace and special characters, and overall aesthetic quality across models and decoding settings. The significance for the AI/ML community lies in testing character-level, formatting-sensitive generation that standard text benchmarks miss. ASCII Bench exposes tokenization and decoding issues (subword vs. character modeling, handling of whitespace and line breaks), sensitivity to temperature/beam search, and the limits of training data for niche symbol patterns. It also highlights evaluation challenges: objective scores (edit distance, line-wise similarity) are imperfect for subjective aesthetics, so human judgment or perceptual metrics are often needed. Practically, this helps researchers tune models for ASCII, code-art, fixed-layout output, and accessibility or legacy-system use cases, while revealing how different architectures and training regimes influence fine-grained, structured text generation.