You Don't Need Blackwells for Realtime Video Generation (darrow8.substack.com)

Recent advancements have led the Hao AI Lab to achieve real-time video generation using their FastH3 model, which leverages innovations like Video Sparse Attention (VSA) to significantly speed up diffusion-based video processing. Initially requiring 49 transformer passes for video generation, the FastH3 model was distilled to just four, allowing it to generate a 14.37-second video in approximately 12.88 seconds on NVIDIA’s B200 GPUs. Further optimizations on Hopper GPUs (H100s and H200s) achieved an impressive speedup of real-time processing, reducing the time to 13.5 seconds on H100s. This breakthrough is significant for the AI/ML community as it demonstrates that real-time video generation does not necessitate the latest hardware advancements, such as NVIDIA’s Blackwell architecture. By optimizing data transfer and model execution, the team discovered that up to 93% of performance gains came from improving computational efficiency rather than raw model speed. This suggests that older equipment may still deliver high performance, stressing the importance of software optimizations in AI model output and accessibility. The work sets a foundation for future research focused on quality metrics and further improvements in video synthesis technologies.