An interactive PyTorch debugger that looks deep inside your neural net (github.com)

A new interactive debugger called Nansense has been introduced for PyTorch, allowing developers to deeply inspect neural network operations during training. Unlike traditional logging tools that analyze post-training metrics, Nansense enables users to visualize activations, gradients, weights, and optimizer states live as they train their models. Key features include the ability to pause and step through batches one at a time, measure the receptive fields of neurons, and conduct experiments like Deep Dream to visualize what each neuron learns over time. With its interactive UI, users can see how specific layers react to inputs and identify optimization bottlenecks such as neuron death and gradient underflow in real-time. This tool is significant for the AI/ML community as it provides a much-needed approach to understanding and interpreting complex models, easing the debugging process. It facilitates deeper insights into model behavior, enabling developers to refine their architectures and training processes more effectively. Moreover, Nansense alleviates the challenge of managing extensive data logging—initializing a live inspection avoids excessive disk usage. With just a few lines of code for integration into training loops, it presents a powerful resource for both novice and experienced practitioners aiming to enhance their model training workflows.