Show HN: LLM Creative Story‑Writing Benchmark V3 (github.com)

The latest release of the LLM Creative Story-Writing Benchmark V3 introduces a rigorous evaluation framework for assessing large language models’ ability to craft engaging fiction while adhering to a structured creative brief. Each model-generated story must seamlessly incorporate ten defined elements—such as character, object, concept, and tone—within a tightly controlled length, ensuring comparability across submissions. Stories are scored on an 18-question rubric split between narrative craftsmanship (character depth, plot coherence, voice, originality, etc.) and faithful integration of required elements. Scores rely on a weighted Hölder mean (p=0.5), which emphasizes weaker aspects to reward consistently strong storytelling rather than isolated strengths. Seventeen leading LLMs were benchmarked using seven independent grader models, combining auto-generated ratings into aggregate scores that highlight differences in literary quality and constraint satisfaction. Top performers include Kimi K2-0905, GPT-5 (medium reasoning), and Qwen 3 Max Preview, which excel not only in plot and prose but also in seamlessly blending assigned elements. The benchmark also reveals nuanced strengths and weaknesses per model via detailed heatmaps and correlation analyses, shedding light on how effectively LLM graders evaluate complex story aspects and element fit. While human validation remains a future step, the strong internal consistency suggests the benchmark reliably measures storytelling capabilities critical for creative AI development. This evaluation marks a meaningful advance in benchmarking LLMs for nuanced, creative language tasks beyond standard NLP benchmarks.