DiffRatio – A One-Step Diffusion Model with SOTA quality and 50% less memory (www.arxiv.org)

Researchers have introduced DIFFRATIO, a novel framework for training one-step diffusion models that achieves state-of-the-art (SOTA) performance while simultaneously reducing memory usage by 50%. Unlike traditional methods that rely on teacher supervision and lengthy pre-training stages, DIFFRATIO directly estimates the score difference between student and data distributions using a single lightweight density-ratio network. This innovation simplifies the training pipeline, minimizes gradient estimation bias, and enhances the quality of one-step image generation across datasets like CIFAR-10 and ImageNet. The significance of DIFFRATIO lies in its potential to streamline the development of diffusion models, making them more accessible for applications in image synthesis, 3D generation, and other domains. By utilizing a direct score estimation approach and eliminating the overhead of teacher models, researchers can achieve high-quality results with fewer computational resources. This advancement not only paves the way for more efficient model development but also holds promise for faster, more effective generative modeling applications in AI and machine learning.