PID: Fast and High-Resolution Latent Decoding with Pixel Diffusion (research.nvidia.com)

A groundbreaking advancement in high-resolution image synthesis has been announced with the introduction of PiD, or Pixel Diffusion Decoder. Unlike traditional latent diffusion models that optimize their decoders primarily for reconstruction, PiD reformulates the decoding process as conditional pixel diffusion. This innovative approach merges decoding and upsampling into a unified generative module, enabling the generation of 2048x2048 pixel images from 512x512 latent representations in under one second on consumer hardware, significantly enhancing both speed and detail. The significance of PiD for the AI/ML community lies in its ability to expedite the decoding process, achieving speed improvements of up to 5.9 times compared to existing methods like SeedVR2, all while delivering superior visual fidelity. Utilizing a lightweight sigma-aware adapter, PiD effectively incorporates noise-corrupted latents, allowing for the early termination of the diffusion process. Furthermore, through techniques such as DMD2, PiD reduces inference to just four steps, marking a substantial leap in efficiency. This advancement not only showcases the potential for faster image generation but also indicates a shift towards more expressive and versatile decoding techniques in high-resolution AI applications.