Detecting model weight tampering in GPU memory (Rome attacks) (stealthium.io)

Research has unveiled Rank-One Model Editing (ROME) attacks, highlighting a critical vulnerability in AI models where malicious Python modules can alter model weights directly in GPU memory, compromising outputs without any detectable file changes. This technique emphasizes the inadequacies of traditional security measures that focus on CPU processes and file activity, as ROME attacks bypass these channels by executing weight modifications solely in the GPU, escaping conventional surveillance. Significantly, Stealthium's detection framework identifies these tampering incidents by linking CUDA kernel launches back to the originating Python code, effectively transforming how runtime security for AI should be approached. By monitoring GPU activity and inspecting model weights in real time, Stealthium can highlight abnormal behavior during the inference process, differentiating legitimate CUDA operations from those associated with ROME attacks. This advancement opens the door for improved defenses against sophisticated threats targeting AI workloads, illustrating the necessity for heightened vigilance in an era where the integrity of AI model predictions is paramount.