Chess engines do weird stuff (girl.surgery)

Recent developments in chess engine optimization have revealed that traditional reinforcement learning (RL) methods may not be necessary for training coexisting models effectively. Once a strong model paired with a search algorithm is established, subsequent engines can leverage this knowledge through a process called model distillation, allowing them to improve without the costly game generation typical of RL training. The Chess engine lc0 utilized this approach, finding that incorporating RL loops led to performance declines. The high Elo ratings achieved by search capabilities alone underscore the efficiency of these new training techniques. Additionally, lc0's innovative strategies involve real-time training enhancements and the application of a technique known as SPSA (Simultaneous Perturbation Stochastic Approximation) to optimize model parameters based on winning outcomes. Even adjustments based on random perturbations have shown to yield significant Elo improvements, highlighting the model's adaptability. Furthermore, lc0's implementation of transformer architecture, enhanced by a system named "smolgen," has produced notable performance gains, demonstrating the versatility and efficacy of these architectures in various applications beyond chess. These advancements signal a potential paradigm shift in how AI models are trained, influencing future approaches in both game theory and broader AI/ML applications.