From Muon to Gradient Clipping: Some Thoughts on QK Stability (MasterGodzilla.github.io)

Keller Jordan and Jeremy Bernstein's innovative Muon optimizer, which operates in function space rather than parameter space, is gaining traction for its potential to enhance deep learning models. However, challenges arise when applying Muon to Transformer models, specifically when adjusting the Query (Q) and Key (K) weight matrices. Direct application can lead to significant instability and potential training collapse. The underlying issue stems from the bilinear nature of the attention mechanism, where independent updates to Q and K do not adequately manage the combined output, potentially causing an "explosion" in their dot product results. To address this problem, the author proposes a modified optimization approach that explicitly constrains the functional output—namely, the attention score matrix—rather than focusing solely on the weight matrices themselves. By enforcing a spectral norm constraint on the change in attention scores, the new framework aims to ensure stability during training by controlling the joint impact of adjustments to Q and K. This refinement not only aligns with Muon's foundational principles but also addresses practical challenges posed by the intricate coupling in attention mechanisms, making it a promising step forward for the AI/ML community in optimizing Transformer-based architectures.