Stochastic Flocks and the Critical Problem of 'Useful' AI (www.techpolicy.press)

Recent developments in AI technology have sparked significant discussion surrounding the rise of "agentic AI," which represents a shift from generative systems that respond to prompts to those that can autonomously create and modify code. This agentic turn allows AI to perform multi-step actions and adapt in real-time, producing software solutions faster than they can be critically evaluated. As companies like OpenAI and Anthropic prepare for initial public offerings, the underlying complexities and risks associated with these advancements become more pronounced, particularly concerns about reliability, accountability, and bias. While the user experience may improve with these systems generating more responsive and capable outputs, critical voices in the AI community emphasize that the foundational issues remain unaddressed. With the proliferation of so-called "slopware"—software produced quickly and often carelessly by these systems—concerns about accountability and the potential for systematic failures grow. As AI-generated code becomes integrated into sensitive applications, the risks of cascading failures and unforeseen consequences increase. This ongoing evolution in AI calls for a heightened dialogue about the implications of such technologies, advocating for a thorough examination of what utility means in the context of AI and the costs and benefits associated with its deployment.