Drawings in untrained neural networks survive the training process (twitter.com)

Researchers have made a fascinating discovery: drawings and images embedded in untrained neural networks can remain recognizable even after the networks have undergone training. For instance, an experiment with a smiley face remained visible post-training in a neural network, as did personal photos used in an MNIST classifier, which still retained features of the original image after completing the training process. This phenomenon was observed across various initialization strategies, weight decay techniques, learning rates, and optimizers. This finding holds significant implications for the AI/ML community, as it suggests that neural networks possess a surprising resilience to the training process, allowing pre-existing information to influence learning outcomes. It raises questions about the stabilization and preservation of information in deep learning models, potentially impacting how researchers design training regimes and architectures. The ability to manipulate and maintain such features could lead to innovative applications in areas like model interpretability, debugging, and even art generation with neural networks, providing exciting avenues for exploration in AI development.