I bought a Grace-Hopper server for €7.5k on Reddit and converted it to a desktop (dnhkng.github.io)

In an adventurous DIY project, a tech enthusiast successfully acquired a Nvidia Grace-Hopper server for €7.5k on Reddit, originally designed for data centers, and modified it into a desktop capable of running 235B parameter models. Typically, the Grace-Hopper platform, which integrates Nvidia’s powerful CPU and GPU technology, is reserved for high-end enterprise use costing upwards of $100,000. The author navigated challenges like converting the liquid-cooled system to air cooling, dealing with noisy operation, and rectifying critical sensor errors. This transformation is significant for the AI/ML community as it demonstrates the potential of repurposing high-performance enterprise hardware for individual use, which could make advanced AI capabilities more accessible to tech-savvy individuals. The project involved detailed technical steps, such as custom cooling solutions and electronic repairs, shedding light on the complexities of running large language models locally. The author's experience also highlights the challenges and intricacies of utilizing such specialized equipment, opening discussions about hardware accessibility and innovation in AI.