Show HN: I asked LLMs to choose between popular developer tools (github.com)

Preseason has launched an open-source benchmark that evaluates which developer tools large language models (LLMs) recommend for building real web applications. By running a standardized set of web-app prompts against various models, the platform collects and analyzes the recommendations for tools and services across multiple categories, including databases, authentication, and analytics. This initiative aims to make the influence of AI-driven developer-tool suggestions more transparent and verifiable, allowing developers to understand which tools LLMs prioritize—knowledge that could significantly impact tool adoption within the development community. The significance of Preseason lies in its potential to shape how developers choose tools, as the recommendations from AI coding assistants often carry more weight than traditional media sources. The project's strict methodology ensures that each tool's recommendation is meticulously documented, capturing whether an LLM suggested a known tool, no tool, or an invalid response. This reproducibility allows developers to challenge or scrutinize the findings, fostering an open exchange of information that could ultimately benefit the broader AI/ML landscape by improving the trustworthiness of AI recommendations in software development.