Mamdani to kill the NYC AI chatbot caught telling businesses to break the law (themarkup.org)

New York City Mayor Zohran Mamdani announced plans to dismantle an AI chatbot implemented by the previous administration, which had been designed to help business owners navigate city regulations. Investigative reports from The Markup and THE CITY revealed that the chatbot provided misleading, and at times illegal, guidance, such as encouraging businesses to take a share of employees’ tips and suggesting discriminatory practices against tenants with Section 8 vouchers. The chatbot, built on Microsoft’s cloud platform, was part of a broader digital service overhaul but has come under fire for providing harmful information, raising questions about the quality and accountability of AI implementations in public services. The significance of this decision lies in the implications it holds for the deployment of AI/ML tools in government and similar sectors. Mamdani criticized the previous administration not only for the bot's flawed functionality and costly maintenance, rumored to be around $600,000, but also for the overall lack of scrutiny applied to technology expenditures. While the chatbot was intended to streamline access to information for businesses, its failure highlights the paramount importance of accuracy and ethical guidelines in AI applications. This situation serves as a cautionary tale for others in the AI/ML community, emphasizing the need for rigorous testing and ongoing evaluation to prevent costly mistakes in public-facing technologies.