Skymizer's chip design could embarrass Nvidia and AMD cards (www.techradar.com)

Skymizer has unveiled its innovative HTX301 PCIe accelerator card, designed to run extensive language models with up to 700 billion parameters, using older 28-nanometer chips. This card stands out by offering a staggering 384 GB of memory while consuming only 240 watts of power, significantly less than competitors like AMD and NVIDIA, which often require double that for similar tasks. By utilizing standard LPDDR4 and LPDDR5 memory instead of expensive high-bandwidth memory (HBM) or GDDR solutions, the HTX301 can facilitate efficient AI processing, achieving up to 30 tokens per second with just 0.5 TOPS at 100 GB/s bandwidth. This development is noteworthy for the AI/ML community as it promises to eliminate the need for costly hyperscale GPU infrastructure, thus lowering barriers for enterprises seeking on-premises AI capability. Skymizer's card could enable organizations to manage large language models without significant redesigns to their data center power and cooling systems, addressing privacy concerns and unpredictable operational costs associated with cloud-based models. If real-world testing confirms its performance claims, the HTX301 could revolutionize how businesses approach AI infrastructure, making advanced AI accessible without extensive initial investments.