Topological Adam: An experiment in energy-stabilized optimization (github.com)

Topological Adam is an experimental optimizer that augments Adam with an internal “energy-stabilization” mechanism inspired by abstract field-coupling ideas (drawn analogously from magnetohydrodynamics but not modeling physics). For each parameter tensor it maintains two auxiliary vectors (α, β) whose dynamics are driven by a coupling current J_t = (α − β) · g_t and updated with rate η and scaling μ0. A joint energy E_t = ½⟨α² + β²⟩ is kept near a target by rescaling the fields, and the usual Adam step is modified with a bounded topological correction: θ ← θ − lr (m̂/√(v̂ + ε) + w_topo · tanh(α − β)). The implementation is available on GitHub and PyPI and plugs into PyTorch with modest runtime overhead, though it uses extra per-parameter memory for α and β. Why it matters: Topological Adam aims to reduce gradient variance and smooth parameter transitions in noisy or unstable training regimes (e.g., RL or small-batch settings). Benchmarks with identical hyperparameters show competitive final accuracy and, in several tasks, improved early/mid-training stability. Recommended tuning ranges are η 0.01–0.10, w_topo 0.001–0.05, and target energy ≈0.5–2.0; defaults mirror Adam for lr, β1, β2. The author emphasizes this is a research tool—promising for stabilization experiments but not a drop-in superior replacement for Adam; users should evaluate memory cost and tune the new hyperparameters for their workloads.