Sana high-resolution image and video generation from NVidia (github.com)

NVIDIA has announced the release of the SANA framework, a comprehensive codebase designed for high-resolution image and video generation. This initiative includes various models such as SANA-1.5, SANA-Video, and SANA-Sprint, each optimizing efficiency in training and inference processes. The highlight is the newly introduced SANA-WM, which features a 2.6 billion parameter controllable world model capable of generating 720p, one-minute videos with six degrees of freedom (6-DoF) camera control. This capability sets a new standard in world modeling and embodied AI. The significance of SANA lies in its ability to facilitate high-quality, efficient generation which is 20 times smaller and 100 times faster than some existing models, such as Flux-12B. Key innovations include the implementation of linear attention mechanisms for improved efficiency, a state-of-the-art decoder-only text encoder for better text-image alignment, and advanced techniques like sCM distillation for one/few-step generation. With its open-source framework and support for low-VRAM devices, SANA aims to democratize access to advanced generative AI, making it more feasible for developers and researchers to experiment and deploy high-resolution models.