Xcena and Samsung's Near Memory Compute CXL Device (chipsandcheese.com)

XCENA has partnered with Samsung to develop the MX1, a groundbreaking CXL memory expansion device that caters to the growing memory demands of machine learning models. Capable of hosting up to 2 TB of DDR5 memory and equipped with a PCIe 6/CXL 3.2 x8 interface offering 128 GB/s bandwidth, the MX1 also includes an impressive onboard compute capability with 3072 RISC-V cores. This design allows the MX1 to perform up to 3 TFLOPS of FP32 computations, making it a promising platform for data-parallel workloads that prioritize memory bandwidth and power efficiency over high single-threaded performance. The significance of the MX1 lies in its ability to mitigate the limitations of traditional memory architectures by integrating compute capabilities directly with memory expansion. It effectively addresses issues of high SSD latency by allowing SSDs to be treated as memory while leveraging DDR5 as a cache. This configuration not only enhances performance but also allows for better utilization of resources by executing computations where data is stored. As technologies like MX1 pave the way for more efficient memory expansion and compute integration, the AI/ML community stands to benefit significantly from improved memory performance and lower operational costs for high-capacity models.