‘Those two jobs need different physics’: Rebellions CEO says training and inference need different chips (www.techradar.com)

Rebellions, an AI hardware startup, has announced a groundbreaking shift in AI chip design by emphasizing the need for distinct architectures tailored for training and inference. Historically, AI chips were designed primarily for training, which requires immense computational power, but with the growing demand for efficient inference—where models are applied in real-world scenarios—the market is changing. Rebellions claims that its racks consume only 16-20kW of power compared to the 120kW typical of leading GPU systems, and they cost about one-third as much. This innovation aims to make AI inference more accessible for enterprises, enabling broader deployment without the need for hyperscale infrastructure. Rebellions' approach addresses fundamental differences in the requirements for training versus inference: training demands high floating-point operations per second (FLOPS), while inference prioritizes efficiency and reliability. Their chiplet-based design keeps memory close to processing units, enhancing performance for large, complex models now common in the industry. By partnering with memory manufacturers like SK Hynix and Samsung, Rebellions ensures that their hardware development aligns with supply capabilities, giving them a competitive advantage. This new direction signals a significant evolution in AI infrastructure, as the industry pivots towards optimizing performance for the growing inference market.