Neural Networks from Scratch in Python: Simpler Than You Think (www.hamza.se)

This piece walks through a compact, from-scratch Python neural network implementation that demystifies how neurons, layers and training work. Starting from the perceptron idea, the author builds an abstract Neuron class (with random weights and bias per incoming ancestor), implements a TanhNeuron (activate = tanh, derivative = 1 - tanh^2), and composes layers into a NeuralNetwork that supports forward inference. The neuron’s get_inputs walks its ancestor graph recursively to compute weighted sums, and inputs are set by marking input-layer neuron activations. The example network shape is [1, 4, 1] trained to approximate sin(x) on [-π, π] using 100 samples, 1,000 epochs and learning rate 0.01. Its training routine implements backpropagation explicitly: compute output errors (target − output), propagate errors backward through hidden layers using downstream weights and activation derivatives, then update weights and biases by adding learning_rate × gradient where gradients combine the error, ancestor activation, and the neuron activation derivative. The write-up is pedagogically valuable—showing end-to-end how initialization, forward pass, error propagation and parameter updates let a tiny network approximate a continuous function—making it a useful, accessible primer for newcomers wanting to understand core ML mechanics before moving to optimized libraries and larger architectures.