Show HN: Lance – image/video generation and understanding in one model (github.com)

ByteDance has unveiled Lance, a cutting-edge unified multimodal model designed for image and video generation, understanding, and editing, all integrated within a single framework. With an efficient architecture of 3 billion parameters, Lance showcases strong performance across various benchmarks for tasks such as text-to-image (t2i) and text-to-video (t2v) generation, as well as image and video editing. This model is notable for being trained from scratch, employing a staged multi-task approach while operating within a budget of 128 A100 GPUs, which emphasizes its capability and efficiency at a relatively lower parameter count compared to other models. The significance of Lance lies not only in its versatile functionality but also in its strong evaluation scores across multiple generation benchmarks, positioning it competitively alongside larger models. Its capabilities extend to nuanced tasks like visual question answering and image-based reasoning, making it a valuable tool for developers and researchers in the AI/ML community. By offering a unified command-line interface and ready-to-run benchmarking scripts, Lance facilitates easy deployment and experimentation, thereby accelerating the research and development of multimodal AI applications.