It Still Can't Do My Job: Four Years of Moving Goalposts (2022–2026) (publicznyprofil.github.io)

The recent analysis, “It Still Can't Do My Job: Four Years of Moving Goalposts (2022–2026),” critically examines the evolving capabilities of AI in software development, highlighting the increasing skepticism surrounding claims of AI's ability to perform complex programming tasks. Initially, tools like ChatGPT struggled with basic coding tasks, prompting a temporary ban on Stack Overflow due to incorrect outputs. As AI models advanced, they began achieving higher exam scores, provoking debates about their actual utility in real-world programming. The release of "Devin," touted as the first AI software engineer, was met with skepticism after an in-depth critique revealed it fell short of expectations, underlining a broader narrative that AI has yet to consistently deliver on its ambitious promises. This ongoing scrutiny is significant for the AI/ML community as it underscores the persistent chasm between impressive demos and practical applications. While AI has shown it can generate boilerplate code and assist in certain programming tasks—reportedly contributing to over a quarter of all new code at Google—many developers remain unconvinced of its readiness for critical production environments. With research indicating that developers may be slower when using AI tools instead of faster, the future of AI in programming remains uncertain. The piece concludes with a challenge for AI: to demonstrate genuine innovation in coding, stability in legacy systems, and autonomous project management, which would mark a true advancement in its capabilities.