The Emperor Pelican Has No Clothes (arcfu.com)

A new benchmark called Design Creativity Bench has been introduced to evaluate the creative capabilities of frontier AI models in UI design tasks. Despite recent advancements in AI's ability to generate functional websites and product UIs, the evaluation reveals that these models, including the top performer OpenAI's GPT-6 Astra, significantly lag behind human designers in creativity. The benchmark assesses three critical dimensions of design creativity: originality, creative range, and appropriateness, using a comprehensive evaluation of 16 AI models across 80 open-ended tasks tailored to different product domains. The findings underscore the persistent gap in creativity between human designers and AI models, with GPT-6 Astra scoring 65.5 compared to a human reference score of 92.3. This work not only highlights the limitations of current AI in making creative design decisions but also aims to establish a reproducible evaluation framework that quantifies design creativity, thus encouraging further research and improvement in AI's creative capabilities. The benchmarks and methodology are made publicly available through GitHub and Hugging Face, fostering transparency and collaboration in the AI/ML community.