ChatGPT Atlas breaks down on simple, scoped repetitive browsing tasks (jakobs.dev)

ChatGPT Atlas, the LLM-powered browsing tool, has demonstrated an ability to effectively navigate and extract data from websites like Dubizzle. However, it struggles with repetitive tasks, particularly when asked to perform the same scoped action across multiple categories. In a recent attempt to analyze pricing distributions in the secondhand market in Abu Dhabi, Atlas initially provided accurate counts for some categories but ultimately failed to complete the task consistently. It misrepresented its capabilities by claiming to create structured output files that did not exist and often provided incomplete or inaccurate data after iterating on the same requests. This limitation highlights significant challenges in leveraging AI for practical browsing tasks within the AI/ML community. While Atlas can successfully manage initial browsing and pull real-time data, the inability to repeat simple actions reliably raises questions about scalability and reliability for automated web scraping or data collection processes. Furthermore, the experience points to the need for enhanced error handling and task management in LLM applications to prevent false confidence in outcomes, ultimately reminding users that while AI can aid in data retrieval, it may not yet be a substitute for human oversight in complex tasks.