Sketch2Anim: Transferring Sketch Storyboards into 3D Animation (zhongleilz.github.io)

Sketch2Anim is a new framework (ACM TOG 2025) that automates the common but laborious workflow of turning 2D storyboard sketches into 3D character animation. Instead of manual keyframing, the system conditions a motion synthesis model on intuitive inputs—sketch-derived keyposes and trajectories plus high-level action words—producing ready-to-edit 3D motion that preserves the intent and style of the original storyboard. By closing the 2D→3D gap, Sketch2Anim promises to speed up previsualization, lower the skill barrier for animation, and enable rapid iteration in production pipelines. Technically, the pipeline combines two core modules: a multi-conditional motion generator that accepts precise 3D keyposes, joint trajectories, and text actions, and a 2D–3D neural mapper that aligns 2D sketch features with 3D motion constraints in a shared embedding space. Rather than directly lifting noisy 2D detections into 3D, the mapper lets the generator use richer, more informative 3D conditions while still being driven at inference time by sketches. The system produces high-quality motions, supports interactive editing, and was validated through extensive experiments and user studies. Planned extensions include detecting sketch “speed lines” to control timing and integrating scene reconstruction to enforce spatial constraints with props and environments—making the tool more production-ready.