US student handcuffed after AI system apparently mistook bag of chips for gun (www.theguardian.com)

A Baltimore high school student, Taki Allen, was handcuffed and searched after an AI-powered gun‑detection system monitoring school cameras reportedly flagged a Doritos bag as a firearm and alerted school officials and police. Officers arrived, ordered Allen to the ground, handcuffed him, and — after finding no weapon — showed him the image that triggered the alarm. Baltimore County began deploying the camera/AI system in its high schools last year; when the model flags a suspicious object it automatically notifies the school and law enforcement. The incident highlights the real-world risks of deploying computer-vision weapon-detection in high-stakes settings: false positives can cause trauma, escalate police encounters, and undermine trust in safety technology. Key technical implications include the need for rigorous, real-world validation of object-detection models, conservative alert thresholds, clear human-in-the-loop verification steps before law enforcement is engaged, and transparency about model performance and failure modes. It also raises broader ethical and operational questions around surveillance, dataset biases, explainability, and accountability when automated alerts directly trigger armed responses in schools.