Comparing GPT-4o vs. GPT-4o-Mini: How Different AI Models Rank the Same Content (lightcapai.medium.com)

A developer ran a small experiment to see how two OpenAI models—GPT-4o and the smaller GPT-4o‑mini—rank the same set of Medium article titles. Using ChatGPT to generate a base script, they scraped article titles, sent identical ranking prompts to both models, and logged the outputs (code and full logs were published). The point wasn’t to name a “winner” but to expose how two models from the same family can diverge when making subjective judgments about writing quality. This matters to the AI/ML community because ranking, curation, and evaluation tasks are often automated and sensitive to subtle model differences. The practical takeaway: smaller, cheaper models (like GPT-4o‑mini) can produce qualitatively different orderings than larger variants, which affects A/B testing, recommender systems, content-moderation heuristics, and research reproducibility. Technically, divergences can arise from differences in capacity, training mixes, tokenization, decoding settings (temperature/top-p), or prompt sensitivity; the experiment underscores the need to validate model choice, tune decoding parameters, and consider ensembling or human-in-the-loop checks when deploying automated subjective judgments.