Minimax M3 weights to be released on Friday (huggingface.co)

Minimax Labs has announced that they will release the weights for their advanced Minimax M3 model this Friday, promising to make strides in addressing community needs. The M3 model has been trained on a massive dataset of 100 trillion tokens, with an estimated parameter count exceeding 500 billion. Notably, it features enhancements such as sparser attention compared to its predecessor, M2.7, which could lead to a significant reduction in the key-value (KV) cache size, dropping by as much as 10 times, thus optimizing performance even for large-scale applications. This open-sourcing initiative is significant for the AI/ML community as it aims to foster collaboration and innovation among developers and researchers. With a community-friendly licensing model in the works, the release is expected to catalyze further research and development, particularly for projects requiring handling large contexts efficiently. The possibility of accommodating up to 256GB unified memory adds to the excitement, as it could enable cutting-edge capabilities that were previously limited by hardware constraints.