Evolution: Training neural networks with genetic selection achieves 81% on MNIST (github.com)

A new evolutionary learning system, GENREG, has achieved an impressive 81% accuracy on the MNIST dataset by using a trust-based approach to train neural networks, bypassing the traditional gradient descent and backpropagation methods. Instead of relying on gradients, GENREG utilizes a population of neural networks that accumulate "trust" based on their performance in recognizing digits. High-performing networks reproduce under evolutionary pressure, introducing mutations to create new generations. This process marks a significant innovation in neural network training as it potentially reduces the reliance on cumbersome hyperparameter tuning and allows for dynamic changes during training, while exploring the weight space more broadly. The implications for the AI/ML community are substantial. GENREG's approach demonstrates that evolutionary selection can discover compact network architectures effectively, achieving competitive performances with significantly fewer parameters compared to conventional models. By emphasizing the importance of population dynamics and stability in evolutionary learning, this technique reveals insights into the minimal viable capacity required for specific tasks. The project also highlights the efficiency of finding solutions under constraints and opens up avenues for future research, including exploring more complex datasets and extending the method to convolutional networks. Overall, GENREG challenges established paradigms in AI training methodologies and could lead to significant advancements in model optimization and application scalability.