Show HN: Deep learning without gradient descent, 500 layers, no skip connections (github.com)

A new deep learning architecture has been introduced by Yuriy N. Bakhvalov, grounded in random function theory and indifference postulates. This architecture aims to address machine learning regression problems without relying on traditional stochastic gradient descent (SGD) techniques. Significant documents detail how this architecture utilizes polyharmonic splines—offering efficient computation and differentiation procedures—placing it as a promising alternative for improving generalization and performance in various machine learning tasks. The architecture has demonstrated impressive results on benchmark datasets, achieving 98.3% accuracy on MNIST without convolutions or data augmentation, and high AUC scores on both the HIGGS and Epsilon datasets. Notably, it can be composed of up to 500 layers without skip connections, indicating a substantial depth and complexity. The provided code repository not only allows for easy implementation but is also geared towards accessibility for those who may only have CPU capabilities. As this work unfolds, it could reshape approaches to deep learning model development, reducing reliance on commonly utilized training methodologies while enhancing computational efficiency.