Visualizing neural network inference in 3D with WebGL and ONNX (www.erikjs.com)

A new real-time neural network visualizer has been developed using WebGL and ONNX, allowing users to observe how MobileNetV2 processes images directly in their browser. This tool aims to demystify the "black box" nature of neural networks by providing a dynamic 3D visualization of the model's inner workings. Users can view signal packets traveling through network layers, layer activations, top predictions with confidence scores, feature maps highlighting detected patterns, and an attention heatmap that illustrates where the model is focusing its analysis—without any data leaving the user's device. The significance of this project lies in its ability to extract and display intermediate activations from the model, thereby offering deeper insights into the neural network’s decision-making process beyond mere final predictions. Utilizing technologies like ONNX Runtime Web for inference and React Three Fiber for rendering, the app achieves responsive performance, with inference times ranging from 100-200ms per image. By combining real-time data visualization with advanced techniques like Class Activation Mapping, the tool not only enhances understanding of neural networks but also enables users to explore and learn from their behavior through direct interaction.