Lance – native supports image and video understanding, generation, and editing (huggingface.co)

ByteDance has announced the launch of Lance, a lightweight unified multimodal model designed for image and video understanding, generation, and editing, all within a single framework. With only 3 billion active parameters, Lance exhibits strong performance across various benchmarks for image and video tasks. It is noteworthy for being trained entirely from scratch using a multi-task approach and a budget involving 128 A100 GPUs. The introduction of Lance is significant for the AI/ML community as it streamlines the workflow for creators and developers by integrating multiple modalities into one model, thus reducing the need for separate tools. Its efficiency at the 3B scale sets a new standard for performance without sacrificing computational resources. This model supports a range of tasks including text-to-image and text-to-video generation, as well as image and video editing, thereby enhancing the potential applications in content creation and media analytics. The project also includes robust command-line interfaces and benchmark scripts, making it accessible for research and practical implementations.