Boosting Wan2.2 I2V Inference on 8xH100s, 56% Faster with Sequence Parallelism (www.morphic.com)

Muhammad Ali Afridi’s optimization suite speeds up Wan2.2 I2V inference on an 8× NVIDIA H100 cluster by combining FlashAttention3, TensorFloat32 (TF32) matmul/cudnn, int8 weight-only quantization, Magcache, torch.compile autotuning, and sequence-parallel scaling. Baseline runs (FlashAttention2) took 250.7s to generate a 1280×720, 81-frame video in 40 steps. Individually, FA3 cut time to 195.1s (−22%), TF32 to 159.6s (−36%), and int8 quantization to 170.2s (−32%). Magcache (E012K2R20) reduced it to 157.1s, and combining Magcache with TF32 fell to 121.6s (−49.4%). Afridi reports that scaling Magcache with sequence parallelism plus compile/autotune pushes overall speedups to roughly the 56% range claimed in the title, while preserving visual quality under conservative Magcache settings. Key technical takeaways: FlashAttention3 and TF32 exploit H100 Hopper tensor cores for large speedups on float32 paths, but TF32 yields no benefit when int8 quantization is used because matmuls operate at reduced precision. Int8_weight_only quantization (via torchao) can also bypass FSDP sharding so both high/low-noise models fit per GPU, trading precision risks for memory and latency gains. Magcache leverages the monotonic decay of residual-magnitude ratios to cache and skip computation; Afridi scales single-GPU magcache to 8 GPUs using sequence parallelism and finds E012K2R20 a good quality/speed sweet spot. Finally, torch.compile (max-autotune-no-cudagraphs) provides additional kernel-level gains but requires warmup/autotuning. The recipe offers a practical, reproducible path for faster high-resolution video generation on multi-H100 rigs with explicit notes on quality trade-offs.