Can AI draw San Francisco from memory? (voygr.tech)

In a fascinating exploration of AI's capabilities, researchers Gurnee and Tegmark assessed six language models from Anthropic, OpenAI, and Google to see if they could accurately "draw" San Francisco from memory based solely on latitude and longitude data. They tested the models on 41,526 points in the city to identify whether it was water, parks, buildings, or streets, validating the AI's responses against OpenStreetMap. The top models—Gemini 4 Argon, GPT-6.1 Sol, and GPT-6 Astra—achieved an impressive accuracy of 86% in categorizing these points overall, though street accuracy remained a challenging area, with the best streets score at just 0.32 for a high reasoning attempt. This study is significant for the AI/ML community as it highlights the models’ strengths and limitations in spatial understanding and memory recall. While the models effectively recognized major geographical features, such as coastlines and significant parks, they struggled with intricate street details. The findings underscore the idea that while AI can provide a high-level overview using learned information, it may not yet replace dedicated geocoding services for specific address queries. The open-source results from this research add valuable insight into AI memory capabilities and performance under varying reasoning levels, advancing the discourse on AI's practical applications in real-world geospatial contexts.