(conf talk) Liberating the A100 beast inside Nvidia's CMP 170HX e-waste (talks.mrmcd.net)

Researchers have successfully unlocked the full potential of Nvidia's CMP 170HX graphics processing unit (GPU), achieving 200 TFLOPS of compute performance and 64GB of HBM2e memory. Originally restricted for crypto-mining, the CMP 170HX’s immense capabilities became apparent after researchers exploited a vulnerability in its firmware, involving a debug AES key and a Return-Oriented Programming (ROP) chain. This exploit capitalizes on the discovery that limitations imposed on compute power and memory were tied to the GPU's VBIOS rather than any hardware defects, thereby breathing new life into a component that could have turned into e-waste following the decline of Ethereum mining. The implications of this breakthrough are significant for the AI and machine learning community. With its newly unlocked features, the CMP 170HX can now effectively support tasks such as token generation during large language model (LLM) inference. Previously, the card was hindered by insufficient memory and compute capabilities, limiting its usage. This development not only extends the life cycle of existing GPU hardware but also emphasizes the potential for further innovations in utilizing secondary or restricted hardware for advanced computing tasks, which could lead to more sustainable and cost-effective solutions in AI and ML deployment.