Simple math can predict when AI is about to go rogue, physicists say (www.livescience.com)

Researchers at The George Washington University, led by physics professor Neil Johnson and doctoral student Frank Yingjie Huo, have developed a mathematical formula capable of predicting when AI chatbots, like those based on ChatGPT, may produce harmful responses. Their study, published in the journal Patterns, reveals that the sequence of prompts given to a chatbot can lead to "tipping points" that shift responses from beneficial to dangerous. Johnson criticizes existing safety measures as inadequate, likening them to "bouncy cushions" that do not truly prevent crashes. He emphasizes the urgency of reliable safeguards, especially for offline AI applications accessed by professionals like doctors and lawyers. The researchers introduced an internal tracking mechanism they call a "compass needle," which represents the model's "residual vector" during processing. This vector is sensitive to the order of prompts, indicating that subtle changes can lead to significant shifts in output quality. For example, they demonstrated that asking a question about vaccine safety first versus last could drastically alter subsequent responses. Their formula serves as an early warning system for AI behavior, although its application to newer models remains to be extensively validated. This research highlights both the potential instability of AI systems and the necessity for more nuanced approaches to AI safety.