Cerebras Deploys Trillion-Parameter AI Models (www.cerebras.ai)

Cerebras has announced the deployment of trillion-parameter AI models using its WSE-3T wafer-scale chips, specifically showcasing the open-source Kimi K3 model with approximately 3 trillion parameters. This marks a significant advancement as it allows the distribution of large models across multiple wafers, enabling high-performance processing speeds of 1000-2000 tokens per second. The innovative approach involves splitting the model into layers that fit the 44 GB of on-chip memory per wafer, with careful mapping of linear-attention and dense layers to maximize efficiency. The technical implications are notable; by storing full layers directly on the wafers, the model avoids unnecessary data movement, enhancing runtime efficiency. Each wafer acts like a stage in an assembly line, processing multiple requests concurrently without idling. This sophisticated pipeline architecture allows for optimal use of resources, supporting a high number of simultaneous requests based on the on-chip memory reserved for key-value caches. Furthermore, the scalability of this approach means that it can easily extend to even larger models exceeding single-chip capacities, positioning Cerebras at the forefront of AI infrastructure innovations.