Tiny's new AI workstation is anything but — packing in 16 TB SSD array and 128 GB GPU memory, but with an eye-watering price tag to boot (www.techradar.com)

Tiny Corp has unveiled the tinybox Red Kimi, a $30,000 AI workstation that boasts four Radeon AI Pro R9700 GPUs with 128 GB GDDR6 memory and a 16 TB NVMe SSD array. This configuration aims to run Moonshot's 2.8-trillion-parameter Kimi K3 model efficiently from a desktop environment rather than a data center. The system facilitates direct GPU access to the SSD storage via its tinygrad driver, designed to optimize performance, while combining power from 2,560 GB/s aggregate bandwidth and a robust 32-core AMD Epyc Genoa processor. For the AI/ML community, the significance of the Red Kimi lies in its approach to handling large models locally, challenging traditional approaches that rely on extensive data center resources. While Tiny claims the machine can process 40 tokens per second, achieving this performance hinges on efficient data caching in GPU memory and potentially lower-precision quantization methods, leaving some uncertainty about the actual performance. The system's unique storage architecture places it competitively against costly alternatives like Apple’s Mac Studio and Nvidia’s DGX Station, marking it as a potentially transformative desktop solution for AI researchers despite its hefty price tag.