GGML GGUF File Format Vulnerabilities (www.databricks.com)

The GGUF file format, utilized for storing and loading model weights in the GGML library, has been found to harbor multiple memory corruption vulnerabilities that could be exploited for code execution. These vulnerabilities arise from insufficient input validation and lack of bounds checking during the parsing of crafted GGUF files, with attack vectors identified around functions like gguf_init_from_file() and gguf_fread_str(). By simply manipulating key-value pairs or providing an unchecked number of array elements, attackers can induce heap overflows, enabling them to overwrite adjacent memory and potentially run malicious code on victim machines. The significance of these vulnerabilities lies in the growing adoption of the GGUF format for distributing machine learning models, particularly with popular frameworks like Llama-2. As these exploits could lead to malware distribution, the findings emphasize the urgent need for robust security protocols in the AI/ML domain. In response, Databricks collaborated with the GGML team to patch these vulnerabilities, underscoring the importance of rigorous security measures in handling machine learning resources. This incident not only highlights the vulnerabilities present in emerging AI technologies but also serves as a call to action for improved security practices within the rapidly evolving field of machine learning.