OpenAI lifts the lid on its in-house Jalapeño chip - with benchmarks claiming it beats Nvidia's GB300 (www.techradar.com)

OpenAI unveiled its in-house Jalapeño chip at the Hot Chips conference, boasting benchmarks that claim 1.5 to 1.9 times more throughput per kilowatt and significantly lower latency compared to Nvidia’s GB200 and GB300 systems. Operating at a power draw of 700W, Jalapeño is designed specifically for inference tasks and showcases efficiency gains vital for AI workloads, though Nvidia still holds an advantage in absolute throughput per package. Notably, Jalapeño's performance metrics come from self-reported data, leading to questions about the reliability of these comparisons. The implications of Jalapeño are significant for the AI/ML community, particularly as it presents a competitive challenge to Nvidia, which currently dominates the inference market. With the chip tailored to minimize data movement and communication delays, and combined with a more efficient B0 stepping slated for future development, OpenAI positions itself as a key player in AI hardware. Analysts emphasize that this could pressure Nvidia’s inference margins, aligning with broader trends as major firms explore custom chip programs. However, the reliance on Nvidia components for training new AI models indicates a complex relationship that could shape future collaborations and competition within the industry.