The famous O3 "GeoGuessr" prompt did not work (www.seangoedecke.com)

A recent exploration of OpenAI's o3 model revealed that the previously acclaimed "GeoGuessr" prompt does not boost performance as expected when identifying geographical locations from images. Kelsey Piper's original discovery had sparked interest in the model's surprising capabilities, but a new evaluation benchmark of 200 images showed that the basic prompt yielded better results than the complex GeoGuessr prompt. The average distance from the correct location was more precise with the former, suggesting that the intricate prompting might not have substantially enhanced the model's geolocation prowess. This finding is significant for the AI/ML community as it emphasizes the necessity of empirical testing to validate claims about model capabilities, rather than relying on anecdotal evidence. The research also highlights challenges in prompt engineering, wherein models can appear to benefit from elaborate instructions even when simpler prompts suffice. Further, this raises questions about the robustness of capabilities across different iterations of AI models, as newer versions like gpt-5.4 and gpt-5.5 did not replicate o3’s geolocation accuracy, illustrating a potential regression in specific functionalities.