OpenAI and Broadcom announce chip designed for LLM inference at scale (arstechnica.com)

OpenAI and Broadcom have unveiled Jalapeño, a new chip specifically engineered for large language model (LLM) inference in data centers. This ASIC (Application-Specific Integrated Circuit) was developed based on extensive collaboration between OpenAI and Broadcom, ensuring it aligns with future LLM needs. The nine-month development cycle emphasizes a tailored approach that aims to optimize performance for current inference demands, marking Jalapeño as the first step in a long-term partnership to enhance chip technology. The significant advancement offered by Jalapeño lies in its promised efficiency; OpenAI reports that initial tests indicate it outperforms existing inference hardware in terms of performance per watt, a crucial metric for data center operations. While comprehensive performance evaluations are still underway, the collaboration between a leading AI research firm and a seasoned silicon supplier signals a pivotal move towards specialized hardware that can better support the scalability and computational needs of future AI models. A detailed technical report is expected in the coming months to further illuminate the chip's capabilities and performance benchmarks.