Show HN: NeuroFlow 55.8x video inference speedup for Vision Transformers PyTorch (github.com)

NeuroFlow has achieved a remarkable 55.8x speedup in video inference for Vision Transformers (ViTs) with a new dynamic routing framework that intelligently processes only meaningful information, reducing computational waste. Traditional ViTs waste significant resources recalculating stationary elements in video feeds; NeuroFlow addresses this by tracking semantic surprise through an Exponential Moving Average of patch embeddings. This allows it to eliminate over 97% of stationary tokens before they reach the encoding stage, drastically reducing the processing time from approximately 678 milliseconds to just 11.9 milliseconds for high-resolution 1792p input. This advancement is significant for the AI/ML community as it optimizes the efficiency of video inference—a crucial aspect of AI applications in various fields, including surveillance and autonomous vehicles. NeuroFlow features three architectural variations, with Architecture C maintaining accuracy without requiring any model weight modifications, achieving 71.55% top-1 accuracy even at 84% token sparsity. Additionally, it demonstrates robust performance across diverse motion inputs, maintaining high accuracy while processing substantial amounts of data faster than ever before. This innovative approach sets new standards in how ViT architectures can be adapted for real-time scenarios with limited computational resources.