Show HN: Zeta – A Dual Whitening Optimizer Saving 20-40% of LLM Training Costs (github.com)

A new dual whitening optimizer called Zeta has been introduced, which promises to reduce large language model (LLM) training costs by an impressive 20-40%. This advancement is especially significant for the AI/ML community as it enhances the training efficiency of various Qwen3 model variants, which range from 0.6B to 8B parameters. The Zeta optimizer reportedly improves convergence rates compared to the traditional AdamW optimizer, achieving gains up to 1.64x with the 1.7B Qwen3 model over 20,000 training steps. This development is noteworthy not just for its potential cost savings but also for its technical implications, particularly as the AI field continues to grapple with the intensive computational demands of training large models. The enhancements offered by Zeta could allow researchers and organizations to lower their carbon footprint and operational expenses while maintaining or improving model performance. As an open-source tool integrated with MindSpeed, Zeta encourages broader collaboration and application among AI researchers, promoting further innovations in optimization techniques.