New Research: AIs are highly inconsistent when recommending brands or products (sparktoro.com)

Recent research has uncovered significant inconsistencies in the way AI tools, such as ChatGPT, Claude, and Google AI, generate brand and product recommendations. The study involved 600 volunteers running 12 different prompts through these AI platforms, producing nearly 3,000 responses. The findings revealed that the likelihood of receiving the same list of recommendations on repeated inquiries is extremely low, with a <1% chance of identical responses from ChatGPT or Google AI. This inconsistency poses serious concerns for businesses investing heavily in AI tracking to gauge brand visibility and rankings. The implications for the AI/ML community are profound. As companies allocate millions to AI-driven marketing strategies, the unreliability of AI-generated recommendations could lead to misguided decisions based on randomized outputs. The study highlights the need for transparency in AI tools, advocating for disclaimers regarding the variability of recommendations. While visibility metrics may still hold value—indicating which brands commonly appear in responses—marketers are cautioned against over-relying on AI ranking orders. This research prompts a reevaluation of how businesses utilize AI tools for decision-making and brand representation, ultimately challenging the narrative of AI infallibility in product recommendations.